Transmitting apparatus, transmitting method, receiving apparatus, and receiving method

ABSTRACT

A transmission apparatus configured to mix, at a mixing rate, pixels of first video data with pixels of peripheral frames of the first video data and obtain second video data at a first frame rate. The mixing rate for each pixel of a frame of the first video data is based on a luminance value of the pixel. The second video data includes frames having a second frame rate lower than the first frame rate. The apparatus encodes the frames having the second frame rate to obtain a basic stream and encodes remaining frames of the second video data to obtain an extended stream, inserts information about the mixing rate into the basic stream and the extended stream in association with each frame, and transmits the basic stream and the extended stream into which the information about the mixing rate has been inserted.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of Japanese Priority PatentApplication JP 2017-120026 filed Jun. 19, 2017, the entire contents ofwhich are incorporated herein by reference.

TECHNICAL FIELD

The present technology relates to a transmitting apparatus, atransmitting method, a receiving apparatus, and a receiving method. Inparticular, the present technology relates to, for example, atransmitting apparatus that transmits moving-image data items at a highframe rate.

BACKGROUND ART

In recent years, cameras that capture images at a high frame rate with ahigh-speed frame shutter have been known. The high frame rate is, forexample, several times, several tens of times, or even several hundredsof times as high as a normal frame rate of, for example, 60 Hz or 50 Hz.

In a service with the high frame rate, it is conceivable to convert,before transmission, a moving-image data item captured with the camerawith the high-speed frame shutter to a moving-image sequence at afrequency lower than an original frequency of the moving-image dataitem. However, although the images captured with the high-speed frameshutter have an advantage of improving motion blurs and achieving imagequality with a high degree of sharpness, the images have a factor thatcauses a problem with image quality in related-art frame interpolationtechnologies used on a receiving-and-replaying side where themoving-image sequence at the frame rate lower than the high frame rateof the moving-image data item to be distributed.

Frame interpolation with the images with the high degree of sharpness,which are captured with the high-speed frame shutter, causes a largerdifference between a case where motion vector search is applicable andotherwise. Thus, the difference therebetween is displayed as noticeableimage degradation. High-load calculation is frequently used to increaseaccuracy of the motion vector search at the time of the frameinterpolation. However, the high-load calculation adversely affects costof the receiver.

The applicant of the present application has previously proposed atechnology for converting the original data items of the images capturedwith the high-speed frame shutter and displaying these images withquality at a certain level or higher on a related-art receiver thatdecodes the data items at the normal frame rate (refer to PatentLiterature 1).

CITATION LIST Patent Literature

PTL 1: PCT Application WO 2015/076277 A1

SUMMARY OF INVENTION Technical Problem

There is a need to satisfactorily transmit moving-image data items at anormal frame rate and a high frame rate.

Solution to Problem

A concept of the present technology lies in a transmitting apparatusincluding circuitry configured to perform processing of mixing, at amixing rate, pixels of each frame of first video data with pixels of oneor more peripheral frames of the first video data and obtain secondvideo data at a first frame rate, wherein the mixing rate for each pixelof the respective frame of the first video data is based on a luminancevalue of the respective pixel. The second video data includes framescorresponding to a second frame rate that is lower than the first framerate, the frames corresponding to the second frame rate being mixed withthe peripheral frames.

In accordance with another concept of the present technology, thecircuitry is further configured to encode the frames corresponding tothe second frame rate to obtain a basic stream and encode remainingframes of the second video data to obtain an extended stream. Thecircuitry inserts information about the mixing rate for each pixel ofthe respective frame of the first video data into the basic stream andthe extended stream in association with the respective frame andtransmits the basic stream and the extended stream into which theinformation about the mixing rate has been inserted.

In accordance with another concept of the present technology, theinformation about the mixing rate for each pixel of the respective frameof the first video data includes plural mixing rates and a correspondingluminance range for at least one of the mixing rates. The basic streamand the extended stream have a Network Abstraction Layer (NAL) unitstructure, and the circuitry is configured to insert a SupplementalEnhancement Information (SEI) NAL unit with the information about themixing rate into the basic stream and the extended stream.

In accordance with another concept of the present technology, thecircuitry is configured to determine, when performing the processing ofmixing the pixels of each frame of the first video data with the pixelsof the one or more peripheral frames of the first video data, the mixingrate for each pixel of the respective frame of the first video databased on a luminance value of the respective pixel.

In accordance with another concept of the present technology, thecircuitry is configured to determine, when performing the processing ofmixing the pixels of each frame of the first video data with the pixelsof the one or more peripheral frames of the first video data, the mixingrate for each pixel of the respective frame of the first video databased on a luminance value of the respective pixel, and based on theluminance values of the pixels of the one or more peripheral frames. Theinformation about the mixing rate for each pixel of the respective frameof the first video data includes a first luminance threshold and asecond luminance threshold, the first luminance threshold and the secondluminance threshold defining the corresponding luminance range for atleast one of the mixing rates. The first frame rate is 120 Hz or 240 Hz,and the second frame rate is 60 Hz.

Another concept of the present technology lies in a transmission methodcomprising performing, by circuitry, processing of mixing, at a mixingrate, pixels of each frame of first video data with pixels of one ormore peripheral frames of the first video data and obtaining secondvideo data at a first frame rate. The mixing rate for each pixel of therespective frame of the first video data is based on a luminance valueof the respective pixel. The second video data includes framescorresponding to a second frame rate that is lower than the first framerate and the frames corresponding to the second frame rate are mixedwith the peripheral frames. The transmission method further includesencoding, by the circuitry, the frames corresponding to the second framerate to obtain a basic stream and encoding remaining frames of thesecond video data to obtain an extended stream. The method furtherincludes inserting, by the circuitry, information about the mixing ratefor each pixel of the respective frame of the first video data into thebasic stream and the extended stream in association with the respectiveframe and transmitting, by the circuitry, the basic stream and theextended stream into which the information about the mixing rate hasbeen inserted.

Another concept of the present technology lies in a reception apparatuscomprising circuitry configured to receive a basic stream and anextended stream, which are obtained by performing processing of mixing,at a mixing rate, pixels of each frame of first video data with pixelsof one or more peripheral frames of the first video data and obtainingsecond video data at a first frame rate. The mixing rate for each pixelof the respective frame of the first video data is based on a luminancevalue of the respective pixel, the second video data include framescorresponding to a second frame rate that is lower than the first framerate, and the frames corresponding to the second frame rate are mixedwith the peripheral frames.

In accordance with another concept of the present technology, the basicstream and the extended stream are obtained by then encoding the framescorresponding to the second frame rate to obtain the basic stream, andencoding remaining frames of the second video data to obtain theextended stream, information about the mixing rate for each pixel of therespective frame of the first video data being included in the basicstream and the extended stream in association with the respective frame.The reception apparatus further includes circuitry configured to, basedon a frame rate capability of a display connected to the receptionapparatus, decode the basic stream to obtain frames at the second framerate or decode the basic stream and the extended stream to obtain thesecond video data, and obtain mixing-released video data at the firstframe rate by performing back mixing processing on the second video dataon a basis of the information about the mixing rate.

In accordance with another concept of the present technology, theinformation about the mixing rate for each pixel of the respective frameof the first video data includes plural mixing rates and a correspondingluminance range for at least one of the mixing rates. The circuitry isconfigured to perform back mixing processing based on the plural mixingrates and the corresponding luminance range for at least one of themixing rates.

Another concept of the present technology lies in a reception methodcomprising receiving, by circuitry, a basic stream and an extendedstream, which are obtained by performing processing of mixing, at amixing rate, pixels of each frame of first video data with pixels of oneor more peripheral frames of the first video data and obtaining secondvideo data at a first frame rate.

In accordance with another concept of the present technology, the mixingrate for each pixel of the respective frame of the first video data isbased on a luminance value of the respective pixel. The second videodata including frames corresponding to a second frame rate that is lowerthan the first frame rate, and the frames corresponding to the secondframe rate are mixed with the peripheral frames.

In accordance with another concept of the present technology, the basicstream and the extended stream are obtained by encoding the framescorresponding to the second frame rate to obtain the basic stream, andencoding remaining frames of the second video data to obtain theextended stream. Information about the mixing rate for each pixel of therespective frame of the first video data is included in the basic streamand the extended stream in association with the respective frame.

In accordance with another concept of the present technology, thereception method further includes, based on a frame rate capability of adisplay connected to the reception apparatus, decoding, by thecircuitry, the basic stream to obtain frames at the second frame rate,or decoding the basic stream and the extended stream to obtain thesecond video data, and obtaining mixing-released video data at the firstframe rate by performing back mixing processing on the second video dataon a basis of the information about the mixing rate.

Another concept of the present technology lies in a reception apparatuscomprising circuitry configured to acquire second video data obtained byperforming processing of mixing, at a mixing rate, pixels of each frameof first video data with pixels of one or more peripheral frames of thefirst video data. The mixing rate for each pixel of the respective frameof the first video data is based on a luminance value of the respectivepixel.

In accordance with another concept of the present technology, thecircuitry is configured to transmit the second video data andinformation about the mixing rate for each pixel of the respective frameof the first video data to an external device via a transfer path, theinformation about the mixing rate for each pixel of the respective frameof the first video data includes plural mixing rates and a correspondingluminance range for at least one of the mixing rates.

In accordance with another concept of the present technology, thecircuitry is configured to respectively insert the information about themixing rate for each pixel of the respective frame into a blankingperiod of the respective frame of the second video data and transmit thesecond video data.

In accordance with another concept of the present technology, thecircuitry is further configured to perform back mixing processing oneach frame of the second video data on a basis of the information aboutthe mixing rate to obtain third video data. The circuitry is configuredto transmit the third video data instead of the second video data whenthe external device does not have a function of the back mixingprocessing.

In accordance with another concept of the present technology, the secondvideo data has a first frame rate, the second video data includes framescorresponding to a second frame rate that is lower than the first framerate, and the frames corresponding to the second frame rate are mixedwith the peripheral frames.

The circuitry is further configured to transmit fourth video data thatincludes the frames corresponding to the second frame rate instead ofthe second video data when a frame rate at which display is able to beperformed by the external device is the second frame rate.

Another concept of the present technology lies in a reception methodcomprising acquiring, by circuitry, second video data obtained byperforming processing of mixing, at a mixing rate, pixels of each frameof first video data with pixels of one or more peripheral frames of thefirst video data. The mixing rate for each pixel of the re spectiveframe of the first video data is based on a luminance value of therespective pixel.

In accordance with another concept of the present technology, thereception method further includes transmitting, by the circuitry, thesecond video data and information about the mixing rate for each pixelof the respective frame of the first video data to an external devicevia a transfer path, the information about the mixing rate for eachpixel of the respective frame of the first video data includes pluralmixing rates and a corresponding luminance range for at least one of themixing rates.

Another concept of the present technology lies in a reception apparatuscomprising circuitry configured to receive second video data obtained byperforming processing of mixing, at a mixing rate, pixels of each frameof first video data with pixels of one or more peripheral frames of thefirst video data. Information about a mixing rate for each pixel of therespective frame of the first video data is received from an externaldevice via a transfer path. The mixing rate for each pixel of therespective frame of the first video data is based on a luminance valueof the respective pixel.

In accordance with another concept of the present technology, thecircuitry is further configured to obtain mixing-released video data byperforming back mixing processing on each frame of the second video dataon a basis of the information about the mixing rate, the informationabout the mixing rate for each pixel of the respective frame of thefirst video data includes plural mixing rates and a correspondingluminance range for at least one of the mixing rates.

Another concept of the present technology lies in a reception methodcomprising receiving, by circuitry, second video data obtained byperforming processing of mixing, at a mixing rate, pixels of each frameof first video data with pixels of one or more peripheral frames of thefirst video data. Information about a mixing rate for each pixel of therespective frame of the first video data is received from an externaldevice via a transfer path, wherein the mixing rate for each pixel ofthe respective frame of the first video data is based on a luminancevalue of the respective pixel.

In accordance with another concept of the present technology, thereception method includes obtaining, by the circuitry, mixing-releasedvideo data by performing hack mixing processing on each frame of thesecond video data on a basis of the information about the mixing rate,the information about the mixing rate for each pixel of the respectiveframe of the first video data includes plural mixing rates and acorresponding luminance range for at least one of the mixing rates.

In accordance with another concept of the present technology, thereceiving unit receives the container containing the video streamobtained by encoding the first moving-image data item at the first framerate. The control unit controls the decoding process and therate-conversion process. The decoding process includes decoding thevideo stream such that the first moving-image data item at the firstframe rate is obtained. The rate-conversion process includes executingthe process of blending the image data items of the peripheral frames ofthe first moving-image data item with the image data items of theprocessing-target frames of the first moving-image data item, theprocessing-target frames corresponding to the second frame rate that islower than the first frame rate, such that the second moving-image dataitem at the second frame rate is obtained.

In this way, pixels of frames at the second frame rate are mixed withpixels of the peripheral frames, that is, under the state of the highshutter-opening rate. Thus, images of this moving image can be smoothlydisplayed in such a manner that the stroboscopic effect is reduced. Inaddition, the image-quality problem as a result of the frameinterpolation process including the low-load calculation in the displayprocess can be avoided.

Note that, in accordance with another concept of the present technology,for example, a receiving apparatus, comprises a receiver configured toreceive a video stream obtained by encoding second video data at a firstframe rate. The receiving apparatus also includes circuitry configuredto control decoding the video stream such that the second video data atthe first frame rate is obtained. The circuitry is further configured tocontrol mixing, at a mixing rate, pixels of each frame of first videodata with pixels of one or more peripheral frames of the first videodata. The second video data includes frames corresponding to a secondframe rate that is lower than the first frame rate, the framescorresponding to the second frame rate being mixed with the peripheralframes, such that a basic stream at the second frame rate is obtained.In accordance with another concept of the present technology, the mixingrate for each pixel of the respective frame of the first video data isbased on a luminance value of the re spective pixel With this, theoriginal texture of the images, such as the high dynamic range (HDR)effect, can be prevented from being impaired by the blending processes.

Advantageous Effects of Invention

In accordance with the concepts of the present technology, moving-imagedata items at a normal frame rate and a high frame rate can besatisfactorily transmitted. Note that, the advantages disclosed hereinare not necessarily limited to those described hereinabove, and all ofthe advantages disclosed herein can be obtained.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing a configuration example of atransmitting-and-receiving system according to a first embodiment.

FIG. 2 is illustrations of an example of a base stream at 60 Hz and anenhanced stream at +60 Hz, which are obtained by a blending process on amoving-image data item at 120 Hz.

FIG. 3 is a graph showing an example of HDR photoelectric conversioncharacteristics.

FIG. 4 is a schematic diagram showing processes in a transmittingapparatus and television receivers.

FIG. 5 is a schematic illustration of an example of blending on atransmitting side and unblending (reverse blending) on a receiving side.

FIG. 6 is schematic illustrations of processes by a pre-processor and apost-processor in a case where a related-art method is employed as amethod of determining blending rates.

FIG. 7 is schematic illustrations of processes by the pre-processor andthe post-processor in a case where a novel method 1 is employed as themethod of determining the blending rates.

FIG. 8 shows a determination logic of a blending process in thepre-processor and in an unblending process (reverse blending process) inthe post-processor in the case where the novel method 1 is employed asthe method of determining the blending rates.

FIG. 9 is schematic illustrations of processes by the pre-processor andthe post-processor in a case where a novel method 2 is employed as themethod of determining the blending rates.

FIG. 10 shows another determination logic of the blending process in thepre-processor and in the unblending process (reverse blending process)in the post-processor in the case where the novel method 2 is employedas the method of determining the blending rates.

FIG. 11 is a block diagram showing a configuration example of thetransmitting apparatus.

FIG. 12 is a block diagram showing a configuration example of thepre-processor in the case where the novel method 1 is employed as themethod of determining the blending rates.

FIG. 13 shows an example of an upper-limit-value table that is used in apixel processing unit.

FIG. 14 shows an example of a lower-limit-value table that is used inthe pixel processing unit.

FIG. 15 is a flowchart showing an example of a procedure for generatingselection signals on a pixel-by-pixel basis in a control unit.

FIG. 16 is a block diagram showing another configuration example of thepre-processor in the case where the novel method 2 is employed as themethod of determining the blending rates.

FIG. 17 is a flowchart showing another example of the procedure forgenerating the selection signals on the pixel-by-pixel basis in thecontrol unit.

FIG. 18 is a table showing a structural example of“Blend_and_range_information SEI message.”

FIG. 19 is a table showing a structural example of“Blend_and_range_information( ).”

FIG. 20 shows contents of main information items in the structuralexample of “Blend_and_range_information( ).”

FIG. 21 is a table showing a structural example of “HFR_descriptor.”

FIG. 22 shows a structural example of a transport stream TS.

FIG. 23 is a block diagram showing a configuration example of atelevision receiver having a decoding capability to process amoving-image data item at a high frame rate (120 Hz).

FIG. 24 is a block diagram showing a configuration example of thepost-processor in the case where the novel method 1 is employed as themethod of determining the blending rates.

FIG. 25 is a flowchart showing an example of a procedure for generatingselection signals on the pixel-by-pixel basis in another control unit.

FIG. 26 is a block diagram showing another configuration example of thepost-processor in the case where the novel method 2 is employed as themethod of determining the blending rates.

FIG. 27 is a flowchart showing another example of the procedure forgenerating the selection signals on the pixel-by-pixel basis in theother control unit.

FIG. 28 is a block diagram showing a configuration example of atelevision receiver having a decoding capability to process amoving-image data item at a normal frame rate (60 Hz).

FIG. 29 is a block diagram showing a configuration example of atransmitting-and-receiving system according to a second embodiment.

FIG. 30 is a flowchart showing an example of a control procedure in acontrol unit (CPU) of a set-top box.

FIG. 31 is a schematic diagram showing processes by the transmittingapparatus, the set-top box, and displays.

FIG. 32A and FIG. 32B are diagrams showing a comparison between a casewhere the display has a function of the reverse blending process(unblending process) and otherwise.

FIG. 33 is a table showing a structural example of “HFR BlendingInfoFrame.”

FIG. 34 shows contents of main information items in the structuralexample of “HFR Blending InfoFrame.”

FIG. 35 is a block diagram shows a configuration example of the set-topbox.

FIG. 36 is a block diagram showing a configuration example of thedisplay compatible with the moving-image data item at the high framerate.

FIG. 37 is a block diagram showing a configuration example of thedisplay compatible with the moving-image data item at the normal framerate.

FIG. 38 is a block diagram showing a configuration example of atransmitting-and-receiving system according to a third embodiment.

FIG. 39 is a schematic diagram showing processes by a transmittingapparatus, a set-top box, and displays.

FIG. 40 is a block diagram showing a configuration example of thetransmitting apparatus.

FIG. 41 is a block diagram showing a configuration example of theset-top box.

FIG. 42 is a block diagram showing a configuration example of anotherpost-processor in the case where the novel method 1 is employed as themethod of determining the blending rates.

FIG. 43 is a block diagram showing another configuration example of theother post-processor in the case where the novel method 2 is employed asthe method of determining the blending rates.

FIG. 44 is a block diagram showing a configuration example of thedisplay compatible with the moving-image data item at the high framerate.

FIG. 45 is a block diagram showing a configuration example of thedisplay compatible with the moving-image data item at the normal framerate.

FIG. 46 is a schematic illustration of an example of the blending on thetransmitting side and the unblending on the receiving side in a generalpattern of the blending process.

FIG. 47A and FIG. 47B each show a method of arranging SEIs in thegeneral pattern of the blending process.

FIG. 48 is a table showing a structural example of“Blend_and_range_information SEI message” in the general pattern of theblending process.

FIG. 49 is a table showing a structural example of “HFR BlendingInfoFrame” in the general pattern of the blending process.

DESCRIPTION OF EMBODIMENTS

Now, embodiments of the present technology (hereinafter, abbreviated as“embodiments”) are described. Note that the description is made in thefollowing order.

-   -   1. First Embodiment    -   2. Second Embodiment    -   3. Third Embodiment    -   4. Modification

1. First Embodiment Transmitting-and-Receiving System

FIG. 1 shows a configuration example of a transmitting-and-receivingsystem 10 according to the first embodiment. Thistransmitting-and-receiving system 10 includes a transmitting apparatus100 and a television receiver 200.

The transmitting apparatus 100 transmits a transport stream TS as acontainer via a broadcast wave. This transport stream TS contains a basestream (base video stream) and an enhanced stream (enhanced videostream) that are obtained by processing a moving-image data item at ahigh frame rate of, for example, 120 Hz or 240 Hz, more specifically, at120 Hz in this embodiment. In this embodiment, the base stream and theenhanced stream each have a NAL unit structure.

The base stream is obtained as follows. Specifically, a moving-imagedata item after a blending process is obtained by executing a process ofblending, at per-frame blending rates based on data levels, image dataitems of peripheral frames of a high-frame-rate moving-image data itembefore the blending with image data items of processing-target frames ofthe high-frame-rate moving-image data item before the blending.

Among image data items of frames of the moving-image data item after theblending process, at least image data items of frames corresponding to anormal frame rate, specifically, corresponding to 60 Hz in thisembodiment are blended with image data items of peripheral frames. Thebase stream is obtained by encoding the image data items of the framescorresponding to the normal frame rate (base frames). Further, theenhanced stream is obtained by encoding image data items of the rest ofthe frames (enhanced frames).

The base stream contains, as access units, the encoded image data itemsof the frames corresponding to the normal frame rate. Further, theenhanced stream contains, as access units, the encoded image data itemsof the enhanced frames corresponding to the high frame rate.

FIG. 2 illustrates an example of a base stream at 60 Hz, which isobtained by execution of the blending process on the moving-image dataitem at 120 Hz, and an enhanced stream at +60 Hz. Frame pairs are eachformed of two frames, one of which constitutes the base stream andanother one of which is subsequent thereto and constitutes an enhancedstream.

In (a) of FIG. 2 , in each of the frame pairs, an image data item of afirst frame, specifically, an image data item of a frame of the basestream is blended with an image data item of the enhanced frame (blendedstate), and this image data item of this frame subsequent thereto of theenhanced stream is not blended with the image data item of the baseframe (unblended state). Further, (b) of FIG. 2 , in each of the framepairs, the image data item of the first frame, specifically, the imagedata item of the frame of the base stream is blended with the image dataitem of the enhanced frame (blended state), and this image data item ofthe frame subsequent thereto of the enhanced stream is also blended withthe image data item of the base frame (blended state).

Information items of the blending rates and a range information item ofeach of the image data items are inserted into a layer of a video (videostream) and/or a layer of a container Specifically, the informationitems of the blending rates contain coefficient sets as many as thenumber of taps of a filter to be used in the blending process. Forexample, when “m”-tap filter by which “m” frames can be blended is used,“m” coefficients are contained in the coefficient set of each of theframes.

Further, the range information item of each of the image data itemscontains information items of a first threshold and a second thresholdsmaller than the first threshold. Further, in this embodiment, theabove-mentioned moving-image data item at 120 Hz is an HDR moving-imagedata item.

FIG. 3 shows an example of HDR photoelectric conversion characteristics.In this graph, the abscissa axis represents luminance, and the ordinateaxis represents transmission code values. The first threshold is a levelvalue “Range_limit_high_value” equivalent to a luminance (100 cd/m²)corresponding to a standard dynamic range (SDR), which is set such thatwhether or not levels of the image data items are within a range ofbrightness reproduction levels as a special-blending-process targetrange is determined. Further, the second threshold is a level value“Range_limit_low_value,” which is set such that whether or not thelevels of the image data items are within a range of dark-partreproduction levels as another special-blending-process target range.

As described above, at the per-frame blending rates in accordance withthe data levels, the image data items of the peripheral frames of themoving-image data item at the high frame rate are blended with the imagedata items of the processing-target frames of the moving-image data itemat the high frame rate. When executing the process of blending the imagedata items of the peripheral frames with the image data items of theprocessing-target frames, the blending rates are determined by using theabove-mentioned level values “Range_limit_high_value” and“Range_limit_low_value” on a pixel-by-pixel basis.

The determination of the blending rates is performed by a novel method 1or a novel method 2. By the novel method 1, the blending rates are eachdetermined based on the image data item of the processing-target frame.Meanwhile, in the novel method 2, the blending rates are each determinedbased not only on the image data item of the processing-target frame butalso on the image data item of the peripheral frame.

In this embodiment, SEINAL units each containing the information item ofthe blending rate (coefficient set) and the range information item ofthe image data item are inserted into the base stream or the enhancedstream. On a receiving side, on the basis of these information items ofthe blending rates and the range information item of each of the imagedata items, at which rate each of the obtained image data items of theframes of each of the base stream and the enhanced stream is blendedwith corresponding one of the image data items of the peripheral framescan be understood.

Referring back to FIG. 1 , the television receiver 200 receives theabove-mentioned transport stream TS that is transmitted via thebroadcast wave from the transmitting apparatus 100. When the televisionreceiver 200 has a decoding capability to process a moving-image dataitem at the normal frame rate (60 Hz), the television receiver 200processes only the base stream contained in the transport stream TS soas to generate the moving-image data item at the normal frame rate, andreproduces its images.

Meanwhile, when the television receiver 200 has a decoding capability toprocess the moving-image data item at the high frame rate (120 Hz), thetelevision receiver 200 processes both the base stream and the enhancedstream contained in the transport stream TS so as to generate themoving-image data item at the high frame rate, and reproduces itsimages. The television receiver 200 acquires the information items ofthe blending rates and the range information item of each of the imagedata items, which are inserted into the layer of the video (videostream) and/or the layer of the container, and executes an unblendingprocess (reverse blending process) with use of these information items.

In this case, the television receiver 200 executes a decoding process onthe base stream so as to generate the image data items of the framescorresponding to the normal frame rate, and executes a decoding processon the enhanced stream so as to generate the image data items of theenhanced frames corresponding to the high frame rate. Then, thetelevision receiver 200 executes the unblending process (reverseblending process) based on the information items of the blending rates(coefficient sets) with use of the image data items of the framescorresponding to the normal frame rate and use of the image data itemsof the enhanced frames corresponding to the high frame rate. In thisway, a moving-image data item at a frame rate as high as that before theblending process is obtained.

FIG. 4 schematically shows processes by the transmitting apparatus 100and the television receivers 200 (200A and 200B). Note that imagesequences P′(N) and P(N+1) of output from a pre-processor 102 of thetransmitting apparatus 100, and image sequences P′(N) and P(N+1) ofoutput from decoders 204 and 204B of the television receivers 200A and200B, which are the same as each other in time series, may be differentfrom each other in image quality due to processes based on codecs. Amoving-image data item Va at a higher frame rate, which is output from acamera (imaging apparatus) 81, is transmitted to an HFR processor 82.With this, a moving-image data item Vb at the high frame rate (120 Hz)is obtained. This moving-image data item Vb is input as a moving-imagedata item P to the transmitting apparatus 100.

In the transmitting apparatus 100, the pre-processor 102 executes theblending process on image data items of frames of the moving-image dataitem P. With this, image data items P′(N) of the frames corresponding tothe normal frame rate, and image data items P(N+1) of the enhancedframes corresponding to the high frame rate are obtained. Note that, inthis embodiment, the image data items P(N+1) are not subjected to theblending with image data items of peripheral frames.

In the transmitting apparatus 100, an encoder 103 executes the encodingprocess on the image data items P′(N) and P(N+1). With this, a basestream STb and an enhanced stream STe are obtained. These streams STband STe are transmitted from the transmitting apparatus 100 to thetelevision receiver 200. Note that the information items of the blendingrates of the frames and the range information items of the image dataitems of the frames are associated respectively with the image dataitems of the frames, and are inserted into these streams STb and STe.

In the television receiver 200A being compatible with the high framerate, that is, having the decoding capability to process themoving-image data item at the high frame rate, the decoder 204 executesthe decoding process on the two streams STb and STe. With this, theimage data items P′(N) of the frames corresponding to the normal framerate, and the image data items P(N+1) of the enhanced framescorresponding to the high frame rate are obtained.

Further, in the television receiver 200A, a post-processor 205 executesthe un-blending process (reverse blending process) on the imageinformation items P′(N) and P(N+1) on the basis of the information itemsof the blending rates of the frames and the range information item ofeach of the image data items. With this, a moving-image data item R atthe high frame rate (120 Hz) as high as that of the moving-image dataitem P on a transmitting side. This moving-image data item R is used asit is as a displaying moving-image data item, or converted to the sameby being increased in frame rate through frame interpolation in amotion-compensated frame interpolation (MCFI) unit 206.

Meanwhile, in the television receiver 200B being compatible with thenormal frame rate, that is, having the decoding capability to processthe moving-image data item at the normal frame rate, the decoder 204Bexecutes the decoding process on the stream STb. With this, the imagedata items P′(N) of the frames corresponding to the normal frame rateare obtained. Then, in the television receiver 200B, a moving-image dataitem including the image data items P′(N) of the frames corresponding tothe normal frame rate is used as it is as a displaying moving-image dataitem, or converted to the same by being increased in frame rate throughframe interpolation in a motion-compensated frame interpolation (MCFI)unit 206B.

FIG. 5 schematically illustrates an example of the blending on thetransmitting side and the unblending on the receiving side. This examplecorresponds to the example in (a) of FIG. 2 , specifically, a picture“N” and a picture “N+1” form a frame pair, and a picture “N+2” and apicture “N+3” form another frame pair. Note that, in the illustratedexample, objects Oa and Ob are static objects, and an object Oc is amoving object.

In each of the frame pairs, by the blending process on the transmittingside, an image data item of a first frame, specifically, an image dataitem of a frame of the base stream is blended with an image data item ofthe enhanced frame (blended state), and this image data item of thisframe subsequent thereto of the enhanced stream is not blended with theimage data item of the base frame (unblended state). Further, theblended state is canceled by the unblending process (reverse blendingprocess) on the receiving side.

FIG. 6 schematically illustrates processes by the pre-processor 102 andthe post-processor 205 in a case where a related-art method is employedas the method of determining the blending rates. (A) of FIG. 6illustrates an image of the picture “N” before the blending process inthe pre-processor 102. (B) of FIG. 6 illustrates an image of the picture“N+1” before the blending process in the pre-processor 102. (C) of FIG.6 illustrates an image of the picture “N” after the blending process inthe pre-processor 102.

A point P(N) in the image of the picture “N” is subjected to thefollowing blending process of Type0 with a point P(N+1) at the sameposition coordinates in the picture “N+1.” A value of P′(N) is obtainedby calculation of an arithmetic mean (weight is ignored) of P(N) andP′(N+1). When a luminance of the point P(N) is high and a luminance ofthe point P(N+1) is not high, a luminance level of P′(N) as a result ofthe blending, that is, as the arithmetic mean, decreases. On thetelevision receiver 200B that is incompatible with the high frame rate,the image obtained as a result of the blending is displayed. Thus, animage to be displayed thereon does not have brightness quality.

Type0 Blending Process

P′(N)=(a/k)*P(N)+(b/k)*P(N+1) (a+b=k)

(D) of FIG. 6 illustrates an image of the picture “N” after theunblending process is executed in the post-processor 205, and (e) ofFIG. 6 illustrates an image of the picture “N+1” that is used in thepost-processor 205. In the television receiver 200A compatible with thehigh frame rate, the post-processor 205 executes the followingunblending process of Type0. With this, a value of P″(N) is obtained. Inthis way, in the television receiver 200A compatible with the high framerate, the brightness quality is restored by the unblending process.

Type0 Unblending Process

P″(N)=k/a*P′(N)−b/a*P(N+1) (a+b=k)

FIG. 7 schematically illustrates processes by the pre-processor 102 andthe post-processor 205 in a case where the novel method 1 is employed asthe method of determining the blending rates. (A) of FIG. 7 illustratesan image of the picture “N” before the blending process in thepre-processor 102. (B) of FIG. 7 illustrates an image of the picture“N+1” before the blending process in the pre-processor 102. (C) of FIG.7 illustrates an image of the picture “N” after the blending process inthe pre-processor 102.

At the time of blending the point P(N) in the picture “N” with the pointP(N+1) at the same position coordinates in the picture “N+1,” pixels areblended while maintaining important quality of luminance andchromaticity. Ranges of levels of pixel values to be maintained are setas the special-blending-process target ranges, and special blending isperformed with respect to pixels within these ranges. In other words,the blending is performed at blending rates different from those in thenormal blending process “Type0 blending process,” or the blending itselfneed not necessarily be performed.

When a pixel value of the point P(N) in the image of the picture “N” isout of either one of the special-blending-process target ranges, thepoint P(N) in the image of the picture “N” is subjected to the followingblending process of Type0 (normal blending process) so as to be blendedwith the point P(N+1) at the same position coordinates in the picture“N+1.” The value of P′(N) is obtained by the calculation of thearithmetic mean (weight is ignored) between P(N) and P(N+1).

Type0 Blending Process

P′(N)=(a/k)*P(N)+(b/k)*P(N+1) (a+b=k)

Meanwhile, when the pixel value of the point P(N) in the image of thepicture “N” is within one of the special-blending-process target ranges,the point P(N) in the image of the picture “N” is subjected to thefollowing blending process of Type1 (special blending process) so as tobe blended with the point P(N+1) at the same position co ordinates inthe picture “N+1.” The value of P′(N) is obtained by the calculation ofthe arithmetic mean (weight is ignored) between P(N) and P(N+1). Withthis, an effect of the blending can be obtained in a normalluminance/chromaticity range, and reception and reproduction can beperformed without impairing sharpness in bright parts and dark parts.

Type1 Blending Process

P′(N)=(d/m)*P(N)+(d/m)*P(N+1) (c+d=m)P′(N)=P(N)(in a case where c=m is satisfied)

(D) of FIG. 7 illustrates an image of the picture “N” after theunblending process in the post-processor 205, and (e) of FIG. 7illustrates an image of the picture “N+1” that is used in thepost-processor 205.

In the television receiver 200A compatible with the high frame rate,when a pixel value of the point P′(N) is out of either one of thespecial-blending-process target ranges, or when the pixel value of thepoint P′(N) is within one of the special-blending-process target ranges,and at the same time, when a pixel value of the point P(N+1) is withinone of the special-blending-process target ranges, the post-processor205 executes the following unblending process of Type0. With this, thevalue of P″(N) is obtained.

Type0 Unblending Process

P″(N)=k/a*P′(N)−b/a*P(N+1) (a+b=k)

Meanwhile, in the television receiver 200A compatible with the highframe rate, when the pixel value of the point P′(N) is within one of thespecial-blending-process target ranges, and at the same time, when thepixel value of the point P(N+1) is out of either one of thespecial-blending-process target ranges, the post-processor 205 executesthe following unblending process of Type1. With this, the value of P″(N)is obtained.

Type1 Unblending Process

P″(N)=m/c*P′(N)−d/c*P(N+1) (c+d=m)P″(N)=P′(N) (in the case where c=m is satisfied)

FIG. 8 shows a determination logic of the blending process in thepre-processor 102 and in the unblending process (reverse blendingprocess) in the post-processor 205 in the case where the novel method 1is employed as the method of determining the blending rates as describedabove.

In FIG. 8 , “range_high” indicates the level value“Range_limit_high_value” being the threshold for determining whether ornot the levels of the image data items are within the range of thebrightness reproduction levels as the special-blending-process targetrange. Further, “range_low” indicates the level value“Range_limit_low_value” being the threshold for determining whether ornot the levels of the image data items are within the range of thedark-part reproduction levels as the other special-blending-processtarget range (refer to FIG. 3 ).

In the blending process in the pre-processor 102, when P(N) satisfies“(P(N)>range_high) or (P(N)<range_low),” it is determined that the“Type1 blending process (Type1_blending( )” is executed. A processingresult P′(N) in this case is within one of the special-blending-processtarget ranges. Further, in another case, in the blending process in thepre-processor 102, it is determined that the “Type0 blending process(Type0_blending( ))” is executed. Whether or not processing resultsP′(N) in these cases are within a normal-blending-process target rangedepends on P(N+1).

Further, in the unblending process in the post-processor 205, in a casewhere P′(N) satisfies “(P′(N)>range_high),” when P(N+1) satisfies“(P(N+1)=<range_high),” it is determined that the “Type1 unblendingprocess (Type1_reverse_blending( ))” is executed. In another case, it isdetermined that the “Type0 unblending process (Type0_reverse_blending())” is executed.

Still further, in the unblending process in the post-processor 205, in acase where P′(N) satisfies “(P′(N)<range_low),” when P(N+1) satisfies“(P(N+1)>=range_low),” it is determined that the “Type1 unblendingprocess (Type1_reverse_blending( ))” is executed. In another case, it isdetermined that the “Type0 unblending process (Type0_reverse_blending())” is executed.

Yet further, in the unblending process in the post-processor 205, instill another case, it is determined that the “Type0 unblending process(Type0_reverse_blending( ))” is executed.

FIG. 9 schematically illustrates processes by the pre-processor 102 andthe post-processor 205 in a case where the novel method 2 is employed asthe method of determining the blending rates. (A) of FIG. 9 illustratesan image of the picture “N” before the blending process in thepre-processor 102. (B) of FIG. 9 illustrates an image of the picture“N+1” before the blending process in the pre-processor 102. (C) of FIG.9 illustrates an image of the picture “N” after the blending process inthe pre-processor 102.

At the time of blending the point P(N) in the picture “N” with the pointP(N+1) at the same position coordinates in the picture “N+1,” the pixelsare blended while maintaining the important quality of luminance andchromaticity. The ranges of the levels of the pixel values to bemaintained are set as the special-blending-process target ranges, andthe special blending is performed with respect to the pixels withinthese ranges. In other words, the blending is performed at blendingrates different from those in the normal blending process “Type0blending process,” or the blending itself need not necessarily beperformed.

When the pixel value of the point P(N) in the image of the picture “N”is out of either one of the special-blending-process target ranges, andat the same time, when the pixel value of the point P(N+1) in the imageof the picture “N+1” is out of either one of thespecial-blending-process target ranges, the point P(N) in the image ofthe picture “N” is subjected to the following blending process of Type0(normal blending process) so as to be blended with the point P(N+1) atthe same position coordinates in the picture “N+1.” The value of P′(N)is obtained by the calculation of the arithmetic mean (weight isignored) between P(N) and P(N+1).

Type0 Blending Process

P′(N)=(a/k)*P(N)+(b/k)*P(N+1) (a+b=k)

Meanwhile, when the pixel value of the point P(N) in the image of thepicture “N” is within one of the special-blending-process target ranges,a point P(N)a in the image of the picture “N” is subjected to thefollowing blending process of Type1 (special blending process) so as tobe blended with a point P(N+1) a at the same position coordinates in thepicture “N+1.” The value of P′(N) is obtained by the calculation of thearithmetic mean (weight is ignored) between P(N)a and P(N+1)a.

Type1 Blending Process

P′(N)=(c/m)*P(N)+(d/m)*P(N+1) (c+d=m)P′(N)=P(N) (in the case where c=m is satisfied)

Further, when the pixel value of the point P(N) in the image of thepicture “N” is out of either one of the special-blending-process targetranges, and at the same time, when the pixel value of the point P(N+1)in the image of the picture “N+1” is within one of thespecial-blending-process target ranges, a point P(N)b in the image ofthe picture “N” is subjected to the following blending process of Type2(special blending process) so as to be blended with a point P(N+1)b atthe same position coordinates in the picture “N+1.” The value of P′(N)is obtained by the calculation of the arithmetic mean (weight isignored) between P(N)b and P(N+1)b. With this, reception andreproduction can be performed while obtaining the effect of the blendingin the normal luminance/chromaticity range, and without impairingsharpness in bright parts and dark parts.

Type2 Blending Process

P′(N)=(e/s)*P(N)+(f/s)*P(N+1) (e<f and e+f=s)

(D) of FIG. 9 illustrates an image of the picture “N” after theunblending process in the post-processor 205, and (e) of FIG. 9illustrates an image of the picture “N+1” that is used in thepost-processor 205.

In the television receiver 200A compatible with the high frame rate,when the pixel value of the point P′(N) is out of either one of thespecial-blending-process target ranges, the post-processor 205 executesthe following unblending process of Type0. With this, the value of P″(N)is obtained.

Type0 Unblending Process

P″(N)=k/a*P′(N)−b/a*P(N+1) (a+b=k)

Meanwhile, in the television receiver 200A compatible with the highframe rate, when the pixel value of the point P′(N) is within one of thespecial-blending-process target ranges, and at the same time, when thepixel value of the point P(N+1) is out of either one of thespecial-blending-process target ranges, the post-processor 205 executesthe following unblending process of Type1. With this, the value of P″(N)is obtained.

Type1 Unblending Process

P″(N)=m/c*P′(N)−d/c*P(N+1) (c+d=m)P″(N)=P′(N) (in the case where c=m is satisfied)

Further, in the television receiver 200A compatible with the high framerate, when the pixel value of the point P′(N) is within one of thespecial-blending-process target ranges, and at the same time, when thepixel value of the point P(N+1) is within one of thespecial-blending-process target ranges, the post-processor 205 executesthe following unblending process of Type2. With this, the value of P″(N)is obtained.

Type2 Unblending Process

P″(N)=s/e*P′(N)−f/e*P(N+1) (e+f=s)

FIG. 10 shows another determination logic of the blending process in thepre-processor 102 and in the unblending process (reverse blendingprocess) in the post-processor 205 in the case where the novel method 2is employed as the method of determining the blending rates as describedabove.

In FIG. 10 , “range_high” indicates the level value“Range_limit_high_value” being the threshold for determining whether ornot the levels of the image data items are within the range of thebrightness reproduction levels as the special-blending-process targetrange. Further, “range_low” indicates the level value“Range_limit_low_value” being the threshold for determining whether ornot the levels of the image data items are within the range of thedark-part reproduction levels as the other special-blending-processtarget range (refer to FIG. 3 ).

In the blending process in the pre-processor 102, when P(N) satisfies“(P(N)>range_high) or (P(N)<range_low),” it is determined that the“Type1 blending process (Type1_blending( ))” is executed. A processingresult P′(N) in this case is within one of the special-blending-processtarget ranges. Further, in the blending process in the pre-processor102, when P(N) satisfies “range_low=<P(N)=<range_high,” and at the sametime, when P(N+1) satisfies “(P(N+1)>range_high) or (P(N+1)<range_low),”it is determined that the “Type2 blending process (Type2_blending( ))”is executed. A processing result P′(N) in this case is within one of thespecial-blending-process target ranges. Further, in another case, it isdetermined that the “Type0 (Type0_blending( ))” is executed. Theprocessing result P′(N) in this case is within thenormal-blending-process target range.

Further, in the unblending process in the post-processor 205, when P′(N)satisfies “(P′(N)>range_high),” and at the same time, when P(N+1)satisfies “(P(N+1)>range_high),” it is determined that the “Type2unblending process (Type2_reverse_blending( )” is executed. When P′(N)satisfies “(P′(N)>range_high),” and at the same time, when P(N+1) doesnot satisfy “(P(N+1)>range_high),” it is determined that the “Type1unblending process (Type1_reverse_blending( ))” is executed.

Still further, in the unblending process in the post-processor 205, whenP′(N) satisfies “(P′(N)<range_low),” and at the same time, when P(N+1)satisfies “(P(N+1)<range_low),” it is determined that the “Type2unblending process (Type2_reverse_blending( ))” is executed. When P′(N)satisfies “(P′(N)<range low),” and at the same time, when P(N+1) doesnot satisfy “(P(N+1)<range_low),” it is determined that the “Type1unblending process (Type1_reverse_blending( ))” is executed.

Yet further, in the unblending process in the post-processor 205, inanother case, it is determined that the “Type0 unblending process(Type0_reverse_blending( ))” is executed.

Configuration of Transmitting Apparatus

FIG. 11 shows a configuration example of the transmitting apparatus 100.This transmitting apparatus 100 includes a control unit 101, thepre-processor 102, the encoder 103, a multiplexer 104, and atransmitting unit 105. The control unit 101 controls operations of theunits in the transmitting apparatus 100.

The pre-processor 102 receives the moving-image data item P at the highframe rate (120 Hz), and outputs the image data items P′(N) of theframes corresponding to the normal frame rate (60 Hz), and the imagedata items P(N+1) of the enhanced frames corresponding to the high framerate.

Note that the pre-processor 102 generates the moving-image data itemafter the blending process by executing the process of blending, at theper-frame blending rates based on data levels, the image data items ofthe peripheral frames of the high-frame-rate moving-image data item Pbefore the blending process with the image data items of theprocessing-target frames of the high-frame-rate moving-image data item Pbefore the blending process. In this moving-image data item, the imagedata items P′(N) correspond to the image data items of the framescorresponding to the normal frame rate (60 Hz), and the image data itemsP(N+1) correspond to the image data items of the rest of the frames. Inthis case, at least the image data items P′(N) are blended with theimage data items of the peripheral frames.

FIG. 12 shows a configuration example of the pre-processor 102. Thisexample is a configuration example in the case where the novel method 1is employed as the method of determining the blending rates. Thispre-processor 102 includes a pixel processing unit 120, a frame delayunit 121, coefficient multipliers 122, 123, 125, and 126, adding units124 and 127, a switching unit 128, and an output unit 129.

The pixel processing unit 120 adjusts values of image data items P(N)and P(N+1) of each of the frames of the moving-image data item P suchthat the process of determining whether or not the pixel values arewithin one of the special-blending-process target ranges on the basis ofthe level values “Range_limit_high_value” and “Range_limit_low_value”can be appropriately executed.

FIG. 13 shows an example of an upper-limit-value table that is used inthe pixel processing unit 120. FIG. 14 shows an example of alower-limit-value table that is used in the pixel processing unit 120.These examples are described by way of an example in which the imagedata items are each a ten-bit data item, a lower-limit value is set to64, and an upper-limit value is set to 940.

First, the example of the upper-limit-value table shown in FIG. 13 isdescribed. This table example is an example in a case where“range_limit_high_value” is set to 700. Values 701 to 705 are changed toa value 705, and values 706 to 940 remain unchanged. Similarly, values698 to 700 are changed to a value 698, and values 64 to 697 remainunchanged.

In this case, when the value of P(N) is 705 or more, it is determinedthat the value of P(N) is within one of the special-blending-processtarget ranges, and the pre-processor 102 executes the “Type1 blendingprocess.” In this case, the value of P(N+1) is 64 or more, and hence, asexpressed by the following equation (1), when a blending rate of P(N)and P(N+1) is set to, for example, 255:1, a minimum value of the valueof P′(N) is calculated as 702.P′(N)=(705*255+64*1)/256=702  (1)

Thus, also on the post-processor 205 side, on the basis of the value ofP′(N), it can be similarly determined that the value of P′(N) is withinone of the special-blending-process target ranges. With this, theunblending process (reverse blending process) can be appropriatelyexecuted. Note that, although not described in detail, even in caseswhere the “range_limit_high_value” is set to values other than 700, thesame advantage can be obtained by adjusting the values of the image dataitems P(N) and P(N+1).

Next, the example of the lower-limit-value table shown in FIG. 14 isdescribed. This table example is an example in a case where“range_limit_low_value” is set to 100. Values 96 to 99 are changed to avalue 96, and values 64 to 95 remain unchanged. Similarly, values 100 to102 are changed to a value 102, and values 103 to 940 remain unchanged.

In this case, when the value of P(N) is 96 or less, it is determinedthat the value of P(N) is within another one of thespecial-blending-process target ranges, and the pre-processor 102executes the “Type1 blending process.” In this case, the value of P(N+1)is 940 or less, and hence, as expressed by the following equation (2),when the blending rate of P(N) and P(N+1) is set to, for example, 255:1,a maximum value of the value of P′(N) is calculated as 99.P′(N)=(96*255+940*1)/256=99  (2)

Thus, also on the post-processor 205 side, on the basis of the value ofP′(N), it can be similarly determined that the value of P′(N) is withinthe other one of the special-blending-process target ranges. With this,the unblending process (reverse blending process) can be appropriatelyexecuted. Note that, although not described in detail, even in caseswhere the “range_limit_low_value” is set to values other than 100, thesame advantage can be obtained by adjusting the values of the image dataitems P(N) and P(N+1).

Note that the upper-limit-value table and the lower-limit-value table,which are set separately from each other in the above description, maybe set as a single table.

Referring back to FIG. 12 , the frame delay unit 121 receives the imagedata items P(N) and P(N+1) that are adjusted in value in the pixelprocessing unit 120, and gives a delay of one frame at 120 Hz. Withthis, when the frame delay unit 121 outputs the image data item P(N) ofthe picture “N,” the pixel processing unit 120 has output the image dataitem P(N+1) of the picture “N+1.”

The image data item P(N) that is obtained from the frame delay unit 121is input to the coefficient multiplier 122 and the coefficientmultiplier 125. Further, the image data item P(N+1) that is obtainedfrom the pixel processing unit 120 is input to the coefficientmultiplier 123 and the coefficient multiplier 126.

The coefficient multiplier 122 has a coefficient (a/k) set by thecontrol unit 101, and the image data item P(N) is multiplied by thiscoefficient. Further, the coefficient multiplier 123 has a coefficient(b/k) set by the control unit 101, and the image data item P(N+1) ismultiplied by this coefficient. Output values from the coefficientmultipliers 122 and 123 are added to each other by the adding unit 124.Note that the coefficient multipliers 122 and 123 and the adding unit124 serve as a filter that executes the “Type0 blending process,” andthe image data item P′(N) generated by the “Type0 blending process” isobtained from the adding unit 124.

Further, the coefficient multiplier 125 has a coefficient (c/m) set bythe control unit 101, and the image data item P(N) is multiplied by thiscoefficient. Still further, the coefficient multiplier 126 has acoefficient (d/m) set by the control unit 101, and the image data itemP(N+1) is multiplied by this coefficient. Output values from thecoefficient multipliers 125 and 126 are added to each other by theadding unit 127. Note that the coefficient multipliers 125 and 126 andthe adding unit 127 serve as a filter that executes the “Type1 blendingprocess,” and the image data item P′(N) generated by the “Type1 blendingprocess” is obtained from the adding unit 127.

The image data items P′(N) obtained in the adding units 124 and 127 areinput to the switching unit 128. In response to selection signals fromthe control unit 101 and on the pixel-by-pixel basis, the switching unit128 selectively outputs the image data item P′(N) obtained by the “Type0blending process” from the adding unit 124, or the image data item P′(N)obtained by the “Type1 blending process” from the adding unit 127.

On the basis of the image data item P(N) obtained from the frame delayunit 121 and the preset level values “range_limit_high_value” and“range_limit_low_value,” the control unit 101 generates the selectionsignals on the pixel-by-pixel basis, and transmits these signals to theswitching unit 128.

FIG. 15 is a flowchart showing an example of a procedure for generatingthe selection signals on the pixel-by-pixel basis in the control unit101. First, in Step ST1, the control unit 101 starts the procedure.Then, in Step ST2, the control unit 101 reads a pixel P(N). Next, inStep ST3, the control unit 101 determines whether or not the pixel P(N)is within the special-blending-process target range “(P(N)>range_high)or (P(N)<range_low).”

When the control unit 101 determines that the pixel P(N) is out ofeither one of the special-blending-process target ranges and within thenormal-blending-process target range, in Step ST4, the control unit 101generates a selection signal for selecting the image data item P′(N)obtained by the “Type0 blending process.” Then, in Step ST5, the controlunit 101 terminates the procedure. Meanwhile, when the control unit 101determines that the pixel P(N) is within one of thespecial-blending-process target ranges, in Step ST6, the control unit101 generates a selection signal for selecting the image data item P′(N)obtained by the “Type1 blending process.” Then, in Step ST5, the controlunit 101 terminates the procedure.

Referring back to FIG. 12 , the image data item P′(N) obtained in theswitching unit 128 and the image data item P(N+1) obtained in the pixelprocessing unit 120 are input to the output unit 129. A framesynchronization signal at 60 Hz is supplied to this output unit 129. Insynchronization with this frame synchronization signal and at the framerate of 60 Hz, the output unit 129 outputs the image data item P′(N) ofeach of the frames corresponding to the normal frame rate, and the imagedata item P(N+1) of each of the enhanced frames corresponding to thehigh frame rate.

FIG. 16 shows another configuration example of the pre-processor 102.This example is a configuration example in the case where the novelmethod 2 is employed as the method of determining the blending rates. InFIG. 16 , the parts corresponding to those in FIG. 12 are denoted by thesame reference symbols, and detailed description thereof is omitted asappropriate. This pre-processor 102 includes the pixel processing unit120, the frame delay unit 121, the coefficient multipliers 122, 123,125, and 126, coefficient multipliers 130 and 131, the adding units 124and 127, an adding unit 132, a switching unit 133, and the output unit129.

The image data item P(N) that is obtained from the frame delay unit 121is input to the coefficient multiplier 122, the coefficient multiplier125, and the coefficient multiplier 130. Further, the image data itemP(N+1) that is obtained from the pixel processing unit 120 is input tothe coefficient multiplier 123, the coefficient multiplier 126, and thecoefficient multiplier 131.

The coefficient multiplier 122 has the coefficient (a/k) set by thecontrol unit 101, and the image data item P(N) is multiplied by thiscoefficient. Further, the coefficient multiplier 123 has the coefficient(b/k) set by the control unit 101, and the image data item P(N+1) ismultiplied by this coefficient. The output values from the coefficientmultipliers 122 and 123 are added to each other by the adding unit 124.Note that the coefficient multipliers 122 and 123 and the adding unit124 serve as the filter that executes the “Type0 blending process,” andthe image data item P′(N) generated by the “Type0 blending process” isobtained from the adding unit 124.

Further, the coefficient multiplier 125 has the coefficient (c/m) set bythe control unit 101, and the image data item P(N) is multiplied by thiscoefficient. Still further, the coefficient multiplier 126 has thecoefficient (dim) set by the control unit 101, and the image data itemP(N+1) is multiplied by this coefficient. The output values from thecoefficient multipliers 125 and 126 are added to each other by theadding unit 127. Note that the coefficient multipliers 125 and 126 andthe adding unit 127 serve as the filter that executes the “Type1blending process,” and the image data item P′(N) generated by the “Type1blending process” is obtained from the adding unit 127.

Still further, the coefficient multiplier 130 has a coefficient (e/s)set by the control unit 101, and the image data item P(N) is multipliedby this coefficient. Further, the coefficient multiplier 131 has acoefficient (f/s) set by the control unit 101, and the image data itemP(N+1) is multiplied by this coefficient. Output values from thecoefficient multipliers 130 and 131 are added to each other by theadding unit 132. Note that the coefficient multipliers 130 and 131 andthe adding unit 132 serve as a filter that executes the “Type2 blendingprocess,” and the image data item P′(N) generated by the “Type2 blendingprocess” is obtained from the adding unit 132.

The image data items P′(N) obtained in the adding units 124, 127, and132 are input to the switching unit 133. In response to the selectionsignals from the control unit 101 and on the pixel-by-pixel basis, theswitching unit 133 selectively outputs the image data item P′(N)obtained by the “Type0 blending process” from the adding unit 124, theimage data item P′(N) obtained by the “Type1 blending process” from theadding unit 127, or the image data item P′(N) obtained by the “Type2blending process” from the adding unit 132.

Based on the image data item P(N) obtained from the frame delay unit121, the image data item P(N+1) obtained from the pixel processing unit120, and the preset level values “range_limit_high_value” and“range_limit_low_value,” the control unit 101 generates the selectionsignals on the pixel-by-pixel basis, and transmits these signals to theswitching unit 133.

FIG. 17 is a flowchart showing another example of the procedure forgenerating the selection signals on the pixel-by-pixel basis in thecontrol unit 101. First, in Step ST11, the control unit 101 starts theprocedure. Then, in Step ST12, the control unit 101 reads the pixel P(N)and a pixel P(N+1). Next, in Step ST13, the control unit 101 determineswhether or not the pixel P(N) is within the special-blending-processtarget range “(P(N)>range_high) or (P(N)<range_low).”

When the control unit 101 determines that the pixel P(N) is out ofeither one of the special-blending-process target ranges and within thenormal-blending-process target range, in Step ST14, the control unit 101determines whether or not the pixel P(N+1) is within thespecial-blending-process target range “(P(N+1)>range_high) or(P(N+1)<range_low).”

When the control unit 101 determines that the pixel P(N+1) is out ofeither one of the special-blending-process target ranges and within thenormal-blending-process target range, in Step ST15, the control unit 101generates the selection signal for selecting the image data item P′(N)obtained by the “Type0 blending process.” Then, in Step ST16, thecontrol unit 101 terminates the procedure. Meanwhile, when the controlunit 101 determines that the pixel P(N+1) is within one of thespecial-blending-process target ranges, in Step ST17, the control unit101 generates a selection signal for selecting the image data item P′(N)obtained by the “Type2 blending process.” Then, in Step ST16, thecontrol unit 101 terminates the procedure.

Further, in Step ST13, when the control unit 101 determines that thepixel P(N) is within one of the special-blending-process target ranges,in Step ST18, the control unit 101 generates the selection signal forselecting the image data item P′(N) obtained by the “Type0 blendingprocess.” Then, in Step ST16, the control unit 101 terminates theprocedure.

Referring back to FIG. 16 , the image data item P′(N) obtained in theswitching unit 133 and the image data item P(N+1) obtained in the pixelprocessing unit 120 are input to the output unit 129. The framesynchronization signal at 60 Hz is supplied to this output unit 129. Insynchronization with this frame synchronization signal and at the framerate of 60 Hz, the output unit 129 outputs the image data item P′(N) ofeach of the frames corresponding to the normal frame rate, and the imagedata item P(N+1) of each of the enhanced frames corresponding to thehigh frame rate.

Referring back to FIG. 11 , the encoder 103 executes the encodingprocess on the image data items P′(N) and P(N+1) obtained from thepre-processor 102 so as to generate the base stream STb and the enhancedstream STe. In this case, predictive encoding processes such asH.264/AVC and H.265/HEVC are executed on the image data items P′(N) andP(N+1).

The encoder 103 inserts the information items of the blending rates andthe range information item of each of the image data items into the basestream STb and the enhanced stream STe. On the receiving side, on thebasis of these information items, at which rate each of the obtainedimage data items of the frames of the base stream is blended withcorresponding one of the image data items of the enhanced frames can beunderstood on the pixel-by-pixel basis. With this, the unblendingprocess (reverse blending process) can be appropriately executed.

In this embodiment, the SEINAL unit containing the information item ofthe blending rate and the range information item of the image data itemis inserted into each access unit of each of the base stream STb and theenhanced stream STe. In this case, the encoder 103 inserts newly defined“Blend_and_range_information SEI message” into a part corresponding to“SEIs” in an access unit (AU).

In this embodiment, the SEINAL unit containing the information item ofthe blending rate and the range information item of the image data item,which relate to the blending process of generating the image data itemP′(N), is inserted into, for example, each of the access units of eachof the base stream STb and the enhanced stream STe. Note that it is alsoconceivable to insert the information item of the blending rate and therange information item of the image data item, which relate to thisblending process, only into each of the access units of the base streamSTb, or only into each of the access units of the enhanced stream STe.

FIG. 18 shows a structural example (syntax) of the“Blend_and_range_information SEI message.” “uuid_iso_iec_11578” has anUUID value specified in “ISO/IEC 11578:1996 AnnexA.”“Blend_and_range_information( )” is inserted into a field of“user_data_payload_bytes.”

FIG. 19 shows a structural example (syntax) of“Blend_and_range_information( )” and FIG. 20 shows contents of maininformation items (semantics) in this structural example. An eight-bitfield of “bit_depth_information” indicates a bit width of an encodedpixel. For example, “0” indicates 8 bits, “1” indicates 10 bits, “2”indicates 12 bits, and “3” indicates 16 bits.

A sixteenth-bit field of “range_limit_high_value” indicates the levelvalue of the upper limit of the normal-blending-process target range. Asixteen-bit field of “range_limit_low_value” indicates the level valueof the lower limit of the normal-blending-process target range. Thesedefine the range information item of each of the image data items.

An eight-bit field of “blending_mode” indicates modes of the blendingprocesses. For example, “0x0” indicates a mode0, that is, a mode ofexecuting only the normal blending process in related art. Further, forexample, “0x1” indicates a mode1, that is, a mode of executing theblending processes including the special blending process based on thedetermination of the pixels in the picture “N.” Still further, forexample, “0x2” indicates a mode2, that is, a mode of executing theblending processes including the special blending process based on thedetermination of the pixels in each of the pictures “N” and “N+1.”

When “blending_mode” is “0x0,” there exist eight-bit fields of“type0_blending_coefficient_a” and “type0_blending_coefficient_b.”Further, when “blending_mode” is “0x1,” there exist eight-bit fields of“type0_blending_coefficient_a,” “type0_blending_coefficient_b,”“type1_blending_coefficient_c,” and “type1_blending_coefficient_d.”Still further, when “blending_mode” is “0x2,” there exist eight-bitfields of “type0_blending_coefficient_a,”“type0_blending_coefficient_b,” “type1_blending_coefficient_c,”“type1_blending_coefficient_d,” “type2_blending_coefficient e,” and“type2_blending_coefficient f.”

The eight-bit field of “type0_ blending_coefficient_a” indicates thecoefficient “a” (coefficient for base-layer pixels) in the “Type0blending process” being the normal blending process. The eight-bit fieldof “type0_blending_coefficient_b” indicates the coefficient “b”(coefficient for enhanced pixels) in the “Type0 blending process” beingthe normal blending process.

The eight-bit field of “type1_blending_coefficient_c” indicates thecoefficient “c” (coefficient for the base-layer pixels) in the “Type1blending process” being the special blending process. The eight-bitfield of “type1_blending_coefficient_d” indicates the coefficient “d”(coefficient for the enhanced pixels) the “Type1 blending process” beingthe special blending process.

The eight-bit field of “type2_blending_coefficient_e” indicates thecoefficient “e” (coefficient for the base-layer pixels) in the “Type2blending process” being the other special blending process. Theeight-bit field of “type2_blending_coefficient_f” indicates thecoefficient “f” (coefficient for the enhanced pixels) in the “Type2blending process” being the other special blending process. Theabove-mentioned coefficients correspond to the information items of theblending rates relating to the blending processes.

Referring back to FIG. 11 , the multiplexer 104 packetizes the basestream STb and the enhanced stream STe generated by the encoder 103 intopacketized elementary stream (PES) packets, and packetizes these packetsfurther into transport packets to be multiplexed. In this way, thetransport stream TS as a multiplexed stream is obtained.

Further, the multiplexer 104 inserts the information items of theblending rates and the range information item of each of the image dataitems into the layer of the transport stream TS as the container. Inthis embodiment, newly defined “HFR_descriptors” are inserted into videoelementary stream loops arranged correspondingly to the base stream andthe enhanced stream under a program map table. Note that, in the casewhere the information items of the blending rates and the rangeinformation item of each of the image data items are arranged in the SEIas described above, the information items can be switched on apicture-by-picture basis or a scene-by-scene basis. Further, in a casewhere the information items of the blending rates and the rangeinformation item of each of the image data items are arranged in thedescriptors of the container, the information items can be switched inunits of longer periods, specifically, on a program-by-program basis orin units of divided programs

FIG. 21 shows a structural example (Syntax) of the “HFR_descriptor.”Although not described in detail, this “HFR_descriptor” containsinformation items similar to those of the above-described“Blend_and_range_information SEI message” (refer to FIG. 19 ).Arrangement of the “HFR_descriptor” is advantageous in that, before thereceiver starts the decoding process, types of processes necessary forHFR picture data items can be understood, and preparation for subsequentpost-processes can be completed.

FIG. 22 shows a configuration example of the transport stream TS. Thistransport stream TS contains the two video streams, that is, the basestream STb and the enhanced stream STe. In other words, in thisconfiguration example, there exist a PES packet “video PES1” of the basestream STb, and a PES packet “video PES2” of the enhanced stream STe.

The “Blend_and_range_information SEI message” (refer to FIG. 19 ) isinserted into each of the encoded image data items of the pictures,which are contained in the PES packet “video PES1” and the PES packet“video PES2.” Note that this example is corresponds to the example inwhich the information items of the blending rates and the rangeinformation item of each of the image data items are inserted into eachof the base stream STb and the enhanced stream STe.

Further, the transport stream TS contains, as one of program specificinformation (PSI) items, the program map table (PMT). These PSIs referto information items of to which program the elementary streamscontained in the transport stream TS belong.

The PMT contains “Program_loop” that describes information itemsrelating to an entirety of a program. Further, the PMT containselementary stream loops each containing information items relating tocorresponding one of the video streams. In this configuration example,there exist a video elementary stream loop “video ES1 loop”corresponding to the base stream, and a video elementary stream loop“video ES2 loop” corresponding to the enhanced stream.

In the “video ES1 loop,” not only information items of, for example, astream type and a packet identifier (PID) corresponding to the basestream (video PES1), but also descriptors that describe the informationitems relating to this video stream, such as “hevc_descriptor” and theabove-mentioned “HFR_descriptor” (refer to FIG. 21 ), are also arranged.In a case of the HEVC encoding, a type of this stream is represented by“0x24” corresponding to the base stream.

Further, in the “video ES2 loop,” not only the information items of, forexample, the stream type and the packet identifier (PID) correspondingto the enhanced stream (video PES2), but also the descriptors thatdescribe the information items relating to this video stream, such as“hevc_descriptor” and the above-mentioned “HFR_descriptor” (refer toFIG. 21 ), are also arranged. A type of this stream is represented by“0x25” corresponding to the enhanced stream.

Note that, in the case of the shown example, the HEVC encoding isperformed, but transmission of signaling information items via the“Blend_and_range_information SEI messages” is applicable also to othercodecs. In cases of the other codecs, different descriptors are insertedinto the PMT.

Referring back to FIG. 11 , the transmitting unit 105 modulates thetransport stream TS, for example, in accordance with a modulation schemesuited to broadcasting, such as QPSK and OFDM, and transmits an RFsignal via a transmitting antenna.

An operation of the transmitting apparatus 100 shown in FIG. 11 isbriefly described. The moving-image data item P at the high frame rate(120 Hz) is input to the pre-processor 102. In this pre-processor 102,the moving-image data item P is processed, and the image data item P′(N)of each of the frames corresponding to the normal frame rate (60 Hz),and the image data item P(N+1) of each of the enhanced framescorresponding to the high frame rate are obtained.

In this case, the moving-image data item after the blending process isobtained by executing the process of blending, at the per-frame blendingrates based on data levels, the image data items of the peripheralframes of the moving-image data item P with the image data items of theprocessing-target frames of the moving-image data item P. Then, in thismoving-image data item, the image data items P′(N) correspond to theimage data items of the frames corresponding to the normal frame rate(60 Hz), and the image data items P(N+1) correspond to the image dataitems of the rest of the frames. In this case, the image data itemsP′(N) are obtained by blending the image data items P(N+1) with theimage data items P(N).

In this embodiment, the blending processes in accordance with the datalevels of the image data items P(N) or with the data level of each ofthe image data items P(N) and P(N+1) and on the pixel-by-pixel basis areselectively used. With this, reception and reproduction can be performedwhile obtaining the effect of the blending in the normalluminance/chromaticity range, and without impairing sharpness in brightparts and dark parts.

For example, in the case where the novel method 1 is employed as themethod of determining the blending rates, in accordance with the datalevels of the image data items P(N), the “Type0 blending process” beingthe normal blending process or the “Type1 blending process” being thespecial blending process is used. Further, in the case where the novelmethod 2 is employed as the method of determining the blending rates, inaccordance with the data levels of the image data items P(N) and P(N+1),the “Type0 blending process” being the normal blending process, the“Type1 blending process” being the special blending process, or the“Type2 blending process” being the other special blending process isused.

The image data items P′(N) and P(N+1) obtained in the pre-processor 102are supplied to the encoder 103. In the encoder 103, the encodingprocess is executed on the image data items P′(N) and P(N+1), and thebase stream STb and the enhanced stream STe are generated. In thisencoder 103, for the sake of convenience of the unblending process(reverse blending process) on the receiving side, the information itemsof the blending rates and the range information item of each of theimage data items, which relate to the blending process, are insertedinto the base stream STb and the enhanced stream STe.

The base stream STb and the enhanced stream STe, which are generated inthe encoder 103, are supplied to the multiplexer 104. In the multiplexer104, the base stream STb and the enhanced stream STe are packetized intothe PES packets, and further into the transport packets to bemultiplexed. In this way, the transport stream TS as a multiplexedstream is obtained. Further, in the multiplexer 104, the“HFR_descriptors” each containing the information items of the blendingrates and the range information item of each of the image data items areinserted into the layer of the transport stream TS as the container.

The transport stream TS generated in the multiplexer 104 is transmittedto the transmitting unit 105. In the transmitting unit 105, thistransport stream TS is modulated, for example, in accordance with themodulation scheme suited to the broadcasting, such as QPSK and OFDM, andthe RF signal is transmitted via the transmitting antenna.

Configuration of Television Receiver

FIG. 23 is a configuration example of the television receiver 200Ahaving the decoding capability to process the moving-image data item atthe high frame rate (120 Hz). This television receiver 200A includes acontrol unit 201, a receiving unit 202, a demultiplexer 203, the decoder204, the post-processor 205, the motion-compensated frame interpolation(MCFI) unit 206, and a panel display unit 207. Further, thedemultiplexer 203 extracts section information items contained in thelayer of the transport stream TS, and transmits these information itemsto the control unit 201. In this case, the “HFR_descriptors” (refer toFIG. 21 ) each containing the information items of the blending ratesand the range information item of each of the image data items are alsoextracted.

The control unit 201 controls operations of the units in the televisionreceiver 200A. The receiving unit 202 acquires the transport stream TSby demodulating the RF-modulated signal received via a receivingantenna. The demultiplexer 203 extracts the base stream STb and theenhanced stream STe from the transport stream TS by filtering the PIDs,and supplies these streams to the decoder 204.

The decoder 204 executes the decoding process on the base stream STb soas to generate the image data item P′(N) of each of the framescorresponding to the normal frame rate, and executes the decodingprocess on the enhanced stream STe so as to generate the image data itemP(N+1) of each of the enhanced frames corresponding to the high framerate.

Further, the decoder 204 extracts a parameter set and the SEI that areinserted into each of the access units of each of the base stream STband the enhanced stream STe, and transmits these information items tothe control unit 201. In this case, the “Blend_and_range_information SEImessages” (refer to FIG. 19 ) each containing the information item ofthe blending rate and the range information item of the image data itemare also extracted.

On the basis of the information items of the blending rates and therange information item of each of the image data items, when theunblending process (reverse blending process) is executed, the controlunit 201 is allowed to appropriately determine which type of theunblending processes (reverse blending processes) to apply on thepixel-by-pixel basis, and is allowed to appropriately set filteringcoefficients in accordance with the types the unblending processes(reverse blending processes). In this way, the post-processor 205 isallowed to appropriately perform control described below of thepost-processor 205.

Under control by the control unit 201, the post-processor 205 executesthe unblending processes (reverse blending processes) based on the imagedata items P′(N) and P(N+1) obtained in the decoder 204. With this, theunblended moving-image data item R at the high frame rate is obtained.

FIG. 24 shows a configuration example of the post-processor 205. Thisexample is a configuration example in the case where the novel method 1is employed as the method of determining the blending rates. Thispost-processor 205 includes coefficient multipliers 220, 221, 223, and224, adding units 222 and 225, switching units 226 and 227.

The image data item P′(N) of the picture “N” is input to the coefficientmultiplier 220 and the coefficient multiplier 223. Further, the imagedata item P(N+1) of the picture “N+1” is input to the coefficientmultiplier 221 and the coefficient multiplier 224.

The coefficient multiplier 220 has a coefficient (k/a) set by thecontrol unit 201, and the image data item P′(N) is multiplied by thiscoefficient. Further, the coefficient multiplier 221 has a coefficient(−b/a) set by the control unit 201, and the image data item P(N+1) ismultiplied by this coefficient. Output values from the coefficientmultipliers 220 and 221 are added to each other by the adding unit 222.Note that the coefficient multipliers 220 and 221 and the adding unit222 serve as a filter that executes the “Type0 unblending process”(reverse blending process), and an image data item P″(N) generated bythe “Type0 unblending process” is obtained from the adding unit 222.

The coefficient multiplier 223 has a coefficient (m/c) set by thecontrol unit 201, and the image data item P′(N) is multiplied by thiscoefficient. Further, the coefficient multiplier 224 has a coefficient(−d/e) set by the control unit 201, and the image data item P(N+1) ismultiplied by this coefficient. Output values from the coefficientmultipliers 223 and 224 are added to each other by the adding unit 225.Note that the coefficient multipliers 223 and 224 and the adding unit225 serve as a filter that executes the “Type1 unblending process”(reverse blending process), and an image data item P″(N) generated bythe “Type1 unblending process” is obtained from the adding unit 225.

The image data items P″(N) obtained in the adding units 222 and 225 areinput to the switching unit 226. In response to selection signals fromthe control unit 201 and on the pixel-by-pixel basis, the switching unit226 selectively outputs the image data item P″(N) obtained by the “Type0unblending process” from the adding unit 222, or the image data itemP″(N) obtained by the “Type1 unblending process” from the adding unit225.

On the basis of the image data items P′(N) and P(N+1) and the levelvalues “range_limit_high_value” and “range_limit_low_value” as the rangeinformation item of each of the image data items, the control unit 201generates the selection signals on the pixel-by-pixel basis, andtransmits these signals to the switching unit 226.

FIG. 25 is a flowchart showing an example of a procedure for generatingthe selection signals on the pixel-by-pixel basis in the control unit201. First, in Step ST21, the control unit 201 starts the procedure.Then, in Step ST22, the control unit 201 reads a pixel P′(N) and a pixelP(N+1). Next, in Step ST23, the control unit 201 determines whether ornot the pixel P′(N) is in the special-blending-process target range“(P′(N)>range_high) or (P′(N)<range_low).”

When the control unit 201 determines that the pixel P′(N) is out ofeither one of the special-blending-process target ranges and within thenormal-blending-process target range, in Step ST24, the control unit 201generates the selection signal for selecting the image data item P″(N)obtained by the “Type0 unblending process.” Then, in Step ST25, thecontrol unit 201 terminates the procedure. Meanwhile, when the controlunit 201 determines that the pixel P′(N) is within one of thespecial-blending-process target ranges, the control unit 201 advancesthe procedure to the process of Step ST26.

In Step ST26, the control unit 201 determines whether or not the pixelP(N+1) is within the special-blending-process target range“(P(N+1)>range_high) or (P(N+1)<range_low).” When the control unit 201determines that the pixel P(N+1) is out of either one of thespecial-blending-process target ranges and within thenormal-blending-process target range, in Step ST27, the control unit 201generates the selection signal for selecting the image data item P″(N)obtained by the “Type1 un-blending process.” Then, in Step ST25, thecontrol unit 201 terminates the procedure.

Meanwhile, when the control unit 201 determines that the pixel P(N+1) iswithin one of the special-blending-process target ranges, in Step ST24,the control unit 201 generates the selection signal for selecting theimage data item P″(N) obtained by the “Type0 unblending process.” Then,in Step ST25, the control unit 201 terminates the procedure.

Referring back to FIG. 24 , the image data item P″(N) obtained in theswitching unit 226 and the image data item P(N+1) are input to theswitching unit 227. A frame synchronization signal at 120 Hz is suppliedto this switching unit 227. In synchronization with this framesynchronization signal, the switching unit 227 extracts the unblendedimage data item P″(N) and the image data item P(N+1) alternately to eachother, and outputs the moving-image data item R at the high frame rate(120 Hz).

FIG. 26 shows another configuration example of the post-processor 205.This example is a configuration example in the case where the novelmethod 2 is employed as the method of determining the blending rates. InFIG. 26 , the parts corresponding to those in FIG. 24 are denoted by thesame reference symbols, and detailed description thereof is omitted asappropriate. This post-processor 205 includes the coefficientmultipliers 220, 221, 223, and 224, coefficient multipliers 230 and 231,the adding units 222 and 225, an adding unit 232, a switching unit 233,and the switching unit 227.

The image data item P′(N) of the picture “N” is input to the coefficientmultiplier 200, the coefficient multiplier 223, and the coefficientmultiplier 230. Further, the image data item P(N+1) of the picture “N+1”is input to the coefficient multiplier 221, the coefficient multiplier224, and the coefficient multiplier 231.

The coefficient multiplier 220 has the coefficient (k/a) set by thecontrol unit 201, and the image data item P′(N) is multiplied by thiscoefficient. Further, the coefficient multiplier 221 has the coefficient(−b/a) set by the control unit 201, and the image data item P(N+1) ismultiplied by this coefficient. The output values from the coefficientmultipliers 220 and 221 are added to each other by the adding unit 222.Note that the coefficient multipliers 220 and 221 and the adding unit222 serve as the filter that executes the “Type0 unblending process”(reverse blending process), and the image data item P″(N) generated bythe “Type0 unblending process” is obtained from the adding unit 222.

The coefficient multiplier 223 has the coefficient (m/c) set by thecontrol unit 201, and the image data item P′(N) is multiplied by thiscoefficient. Further, the coefficient multiplier 224 has the coefficient(−d/c) set by the control unit 201, and the image data item P(N+1) ismultiplied by this coefficient. The output values from the coefficientmultipliers 223 and 224 are added to each other by the adding unit 225.Note that the coefficient multipliers 223 and 224 and the adding unit225 serve as the filter that executes the “Type1 unblending process”(reverse blending process), and the image data item P″(N) generated bythe “Type1 unblending process” is obtained from the adding unit 225.

The coefficient multiplier 230 has a coefficient (e/s) set by thecontrol unit 201, and the image data item P′(N) is multiplied by thiscoefficient. Further, the coefficient multiplier 231 has a coefficient(−f/e) set by the control unit 201, and the image data item P(N+1) ismultiplied by this coefficient. Output values from the coefficientmultipliers 230 and 231 are added to each other by the adding unit 232.Note that the coefficient multipliers 230 and 231 and the adding unit232 serve as a filter that executes the “Type2 unblending process”(reverse blending process), and the image data item P″(N) generated bythe “Type2 unblending process” is obtained from the adding unit 232.

The image data items P″(N) obtained in the adding units 222, 225, and232 are input to the switching unit 233. In response to the selectionsignals from the control unit 201 and on the pixel-by-pixel basis, theswitching unit 233 selectively outputs the image data item P″(N)obtained by the “Type0 unblending process” from the adding unit 222, theimage data item P″(N) obtained by the “Type1 unblending process” fromthe adding unit 225, or the image data item P″(N) obtained by the “Type2unblending process” from the adding unit 232.

On the basis of the image data items P′(N) and P(N+1) and the levelvalues “range_limit_high_value” and “range_limit_low_value” as the rangeinformation item of each of the image data items, the control unit 201generates the selection signals on the pixel-by-pixel basis, andtransmits these signals to the switching unit 233.

FIG. 27 is a flowchart showing another example of the procedure forgenerating the selection signals on the pixel-by-pixel basis in thecontrol unit 201. First, in Step ST31, the control unit 201 starts theprocedure. Then, in Step ST32, the control unit 201 reads the pixelsP′(N) and P(N+1). Next, in Step ST33, the control unit 201 determineswhether or not the pixel P′(N) is within the special-blending-processtarget range “(P′(N)>range_high) or (P′(N)<range_low).”

When the control unit 201 determines that the pixel P′(N) is out ofeither one of the special-blending-process target ranges and within thenormal-blending-process target range, in Step ST34, the control unit 201generates the selection signal for selecting the image data item P″(N)obtained by the “Type0 unblending process.” Then, in Step ST35, thecontrol unit 201 terminates the procedure.

Further, in Step ST33, when the control unit 201 determines that thepixel P′(N) is within one of the special-blending-process target ranges,in Step ST36, the control unit 201 determines whether or not the pixelP(N+1) is within the special-blending-process target range“(P(N+1)>range_high) or (P(N+1)<range_low).” When the control unit 201determines that the pixel P(N+1) is out of either one of thespecial-blending-process target ranges and within thenormal-blending-process target range, in Step ST37, the control unit 201generates the selection signal for selecting the image data item P″(N)obtained by the “Type1 unblending process.” Then, in Step ST35, thecontrol unit 201 terminates the procedure.

Still further, in Step ST36, when the control unit 201 determines thatthe pixel P(N+1) is within one of the special-blending-process targetranges, in Step ST38, the control unit 201 generates the selectionsignal for selecting the image data item P″(N) obtained by the “Type2unblending process.” Then, in Step ST35, the control unit 201 terminatesthe procedure.

Referring back to FIG. 26 , the image data item P″(N) obtained in theswitching unit 233 and the image data item P(N+1) are input to theswitching unit 227. The frame synchronization signal at 120 Hz issupplied to this switching unit 227. In synchronization with this framesynchronization signal, the switching unit 227 extracts the unblendedimage data item P″(N) and the image data item P(N+1) alternately to eachother, and outputs the moving-image data item R at the high frame rate(120 Hz).

Referring back to FIG. 23 , the MCFI unit 206 executes amotion-compensated frame interpolation process on the moving-image dataitem R at the high frame rate, which is obtained in the post-processor205. With this, a moving-image data item at a much higher rate isobtained. Note that this MCFI unit 206 may be omitted. The panel displayunit 207 displays images of the moving-image data item R at the highframe rate, which is obtained in the post-processor 205, or images ofthe moving-image data item increased in frame rate in the MCFI unit 206.

The operations in the television receiver 200A shown in FIG. 23 arebriefly described. In the receiving unit 202, the transport stream TS isacquired by demodulating the RF-modulated signal received via thereceiving antenna. This transport stream TS is transmitted to thedemultiplexer 203. In the demultiplexer 203, the base stream STb and theenhanced stream STc are extracted from the transport stream TS byfiltering the PIDs, and are supplied to the decoder 204.

In the decoder 204, the decoding process is executed on the base streamSTb such that the image data item P′(N) of each of the framescorresponding to the normal frame rate is obtained, and the decodingprocess is executed on the enhanced stream STe such that the image dataitem P(N+1) of each of the enhanced frames corresponding to the highframe rate is obtained. These image data items P′(N) and P(N+1) aresupplied to the post-processor 205.

Further, in the decoder 204, the parameter set and the SEI that areinserted into each of the access units of each of the base stream STband the enhanced stream STe are extracted and transmitted to the controlunit 201. In this case, the “Blend_and_range_information SEI messages”(refer to FIG. 19 ) each containing the information item of the blendingrate and the range information item of the image data item are alsoextracted.

In the control unit 201, on the basis of the information items of theblending rates and the range information item of each of the image dataitems, when executing the unblending process (reverse blending process),which type of the unblending processes (reverse blending processes) toapply on the pixel-by-pixel basis is allowed to be appropriatelydetermined. In addition, the filtering coefficients in accordance withthe types the unblending processes (reverse blending processes) areallowed to be appropriately set.

In the post-processor 205, under the control by the control unit 201,the unblending processes (reverse blending processes) are executed onthe basis of the image data items P′(N) and P(N+1) obtained in thedecoder 204. With this, the unblended moving-image data item R at thehigh frame rate is obtained.

For example, in the case where the novel method 1 is employed as themethod of determining the blending rates, in accordance with the datalevels of the image data item P′(N), the “Type0 unblending process”being the normal unblending process or the “Type1 unblending process”being the special unblending process is used. Further, in the case wherethe novel method 2 is employed as the method of determining the blendingrates, in accordance with the data levels of the image data items P′(N)and P(N+1), the “Type0 unblending process” being the normal unblendingprocess, the “Type1 unblending process” being the special unblendingprocess, or the “Type2 unblending process” being the other specialunblending process is used.

The moving-image data item R at the high frame rate, which is obtainedin the post-processor 205, or the moving-image data item increased inframe rate in the MCFI unit 206 is supplied to the panel display unit207. The images of these moving-image data items are displayed on thepanel display unit 207.

FIG. 28 is a configuration example of the television receiver 200Bhaving the decoding capability to process the moving-image data item atthe normal frame rate (60 Hz). This television receiver 200B includes acontrol unit 201B, a receiving unit 202B, a demultiplexer 203B, thedecoder 204B, the MCFI unit 206B, and a panel display unit 207B.

The control unit 201B controls operations of the units in the televisionreceiver 200B. The receiving unit 202B acquires the transport stream TSby demodulating the RF-modulated signal received via the receivingantenna. The demultiplexer 203B extracts the base stream STb from thetransport stream TS by filtering the PIDs, and supplies this stream tothe decoder 204B. The decoder 204B executes the decoding process on thebase stream STb so as to generate the image data item P′(N) of each ofthe frames corresponding to the normal frame rate.

The MCFI unit 206B executes the motion-compensated frame interpolationprocess on these image data items P′(N) so as to generate a moving-imagedata item at a much higher rate. Note that this MCFI unit 206B may beomitted. The panel display unit 207B displays images of the moving-imagedata item at the normal frame rate (image data items P′(N)), which isobtained in the decoder 204B, or of the moving-image data item increasedin frame rate in the MCFI unit 206B.

The operations in the television receiver 200B shown in FIG. 28 arebriefly described. In the receiving unit 202B, the transport stream TSis acquired by demodulating the RF-modulated signal received via thereceiving antenna. This transport stream TS is transmitted to thedemultiplexer 203B. In the demultiplexer 203B, the base stream STb isextracted from the transport stream TS by filtering the PIDs, and issupplied to the decoder 204B.

In the decoder 204B, the decoding process is executed on the base streamSTb such that the image data item P′(N) of each of the framescorresponding to the normal frame rate is obtained. The moving-imagedata item at the normal frame rate, which is obtained in the decoder204B, or the moving-image data item increased in frame rate in the MCFIunit 206B is supplied to the panel display unit 207B. The images ofthese moving-image data items are displayed on the panel display unit207B.

As described hereinabove, in the transmitting-and-receiving system 10shown in FIG. 1 , among the image data items of the frames of themoving-image data item at the high frame rate, at least the image dataitems of the frames corresponding to the normal frame rate are blendedwith the image data items of the peripheral frames, that is, under astate of a high shutter-opening rate. The base stream STb to betransmitted is obtained by encoding these image data items of the framescorresponding to the normal frame rate.

Thus, as for the television receiver 200B having the decoding capabilityto process the moving-image data item at the normal frame rate, themoving-image data item at the normal frame rate is obtained byprocessing the base stream STb, and images of the moving image can besmoothly displayed. In addition, an image-quality problem as a result ofthe frame interpolation process including low-load calculation in thedisplay process can be avoided.

Further, in the transmitting-and-receiving system 10 shown in FIG. 1 ,the image data item of each of the frames is blended with the image dataitem of corresponding one of the peripheral frames at the blending ratein accordance with the data level. Thus, the original texture of theimages, such as a high dynamic range (HDR) effect, can be prevented frombeing impaired by the blending processes.

Still further, in the transmitting-and-receiving system 10 shown in FIG.1 , in addition to the base stream STb, the enhanced stream STe isobtained by encoding the image data items of the rest of the frames. Thebase stream STb and the enhanced stream STe are transmitted with theinformation items of the blending rates (coefficient sets) of the framesand the range information item of each of the image data items, whichare associated respectively with the image data items of the frames,being inserted into these streams. Thus, on the receiving side, on thebasis of these information items, the unblended moving-image data itemcan be easily and appropriately obtained by executing the unblendingprocesses (reverse blending processes) on the moving-image data itemobtained by decoding the base stream and the enhanced stream.

Note that the present technology is not limited to the above-describedexample of the transmitting-and-receiving system 10 shown in FIG. 1 , inwhich the base stream STb and the enhanced stream STe are transmitted tothe receiving side with the information items of the blending rates ofthe frames and the range information item of each of the image dataitems being inserted therein. The information items of the blendingrates of the frames and the range information item of each of the imagedata items may be provided with other units to the receiving side.

2. Second Embodiment Transmitting-and-Receiving System

In the example in the above-described embodiment, thetransmitting-and-receiving system 10 includes the transmitting apparatus100 and the television receiver 200. However, the present technology isapplicable is applicable to other configurations of thetransmitting-and-receiving system. The television receiver 200 may bereplaced with a set-top box and a display that are connected to eachother via digital interfaces (multimedia interfaces) such as ahigh-definition multimedia interface (HDMI). Note that the “HDMI” is atrademark.

FIG. 29 shows a configuration example of a transmitting-and-receivingsystem 10A. This transmitting-and-receiving system 10A includes thetransmitting apparatus 100, a set-top box (STB) 200-1, and a display200-2. The set-top box (STB) 200-1 and the display 200-2 are connectedto each other via the HDMI.

The transmitting apparatus 100 is the same as the transmitting apparatus100 in the transmitting-and-receiving system 10 shown in FIG. 1 , andhence description thereof is omitted. The set-top box 200-1 receives thetransport stream TS that is transmitted via the broadcast wave from thetransmitting apparatus 100.

In a case where the display 200-2 is compatible with the moving-imagedata item at the high frame rate (120 Hz), the set-top box 200-1processes both the base stream STb and the enhanced stream STe containedin the transport stream TS. With this, the image data items P′(N) andP(N+1) are obtained.

In a case where the display 200-2 has the function of the unblendingprocess (reverse blending process), the set-top box 200-1 transmits, tothe display 200-2 via the HDMI transmission path, the image data itemsP′(N) and P(N+1) of each of the frames, which have been subjected to theblending process, the information items of the blending rates(coefficient sets), and the range information item of each of the imagedata items.

Further, in a case where the display 200-2 does not have the function ofthe unblending process (reverse blending process), the set-top box 200-1executes, on the basis of the information items of the blending rates(coefficient sets), the unblending process (reverse blending process) onthe image data items P′(N) and P(N+1) that have been subjected to theblending process. With this, the unblended moving-image data item R atthe high frame rate is obtained. Then, the set-top box 200-1 transmitsthis moving-image data item R at the high frame rate to the display200-2 via the HDMI transmission path.

Meanwhile, in a case where the display 200-2 is compatible with themoving-image data item at the normal frame rate (60 Hz), the set-top box200-1 processes only the base stream STb contained in the transportstream TS. With this, the image data item P′(N) of each of the framescorresponding to the normal frame rate is obtained. Then, the set-topbox 200-1 transmits these image data items P′(N) to the display 200-2via the HDMI transmission path.

The set-top box 200-1 being a source apparatus acquires EDTD from thedisplay 200-2 being a sink apparatus, and determines whether or not thedisplay 200-2 is compatible with the moving-image data item at the highframe rate (120 Hz), and determines whether or not the display 200-2 hasthe function of the unblending process (reverse blending process).

FIG. 30 is a flowchart showing an example of a control procedure in acontrol unit (CPU) of the set-top box 200-1. First, in Step ST41, thecontrol unit starts the control procedure. Then, in Step ST42, thecontrol unit checks the EDTD read out from the display 200-2. Next, inStep ST43, the control unit determines whether or not the display 200-2is compatible with the moving-image data item at the high frame rate(120 Hz).

When the display 200-2 is incompatible therewith, in Step ST44, thecontrol unit decodes only the base stream STb, and transmits the imagedata item P(N) of each of the frames corresponding to the normal framerate (60 Hz) to the display 200-2. After the process of this Step ST44,in Step ST45, the control unit terminates the control procedure.

Further, in Step ST43, when the control unit determines that the display200-2 is compatible with the moving-image data item at the high framerate, in Step ST46, the control unit decodes both the base stream STband the enhanced stream STe.

Then, in Step ST47, the control unit determines whether or not thedisplay 200-2 has the function of the unblending process (reverseblending process). When the display 200-2 does not have the function ofthe unblending process, in Step ST48, the control unit determinesexecution of the unblending process on the set-top box 200-1 side, andtransmits the unblended moving-image data item R at the high frame rateto the display 200-2. After the process of this Step ST48, in Step ST45,the control unit terminates the control procedure.

Further, in Step ST47, when the control unit determines that the display200-2 has the function of the unblending process, in Step ST49, thecontrol unit determines execution of the unblending process on thedisplay 200-2 side, and transmits, to the display 200-2 via the HDMItransmission path, the image data items P′(N) and P(N+1) that have beensubjected to the blending process, the information items of the blendingrates (coefficient sets), and the range information item of each of theimage data items. After the process of this Step ST49, in Step ST45, thecontrol unit terminates the control procedure.

FIG. 31 schematically shows processes by the transmitting apparatus 100,the set-top box 200-1, and the display 200-2. Note that the imagesequences P′(N) and P(N+1) of the output from the pre-processor 102 ofthe transmitting apparatus 100, and the image sequences P′(N) and P(N+1)of the output from the decoder 204 of the set-top box 200-1, which arethe same as each other in time series, may be different from each otherin image quality due to the processes based on the codecs.

The transmitting apparatus 100 is the same as that described withreference to FIG. 4 , and hence description thereof is omitted. In theset-top box 200-1, in a case where a display 200-2A compatible with themoving-image data item at the high frame rate (120 Hz) is connectedthereto, the decoder 204 executes the decoding process on the twostreams STb and STe. With this, the image data item P′(N) of each of theframes corresponding to the normal frame rate, and the image data itemsP(N+1) of each of the enhanced frames corresponding to the high framerate are obtained.

Further, in a case where the display 200-2A has the function of theunblending process (reverse blending process), from the set-top box200-1, the image data items P′(N) and P(N+1), the information items ofthe blending rates (coefficient sets), and the range information item ofeach of the image data items are transmitted to the display 200-2A viathe HDMI transmission path. In the case of the shown example, thedisplay 200-2A includes the post-processor 205, and the display 200-2Ahas the function of the unblending process (reverse blending process).Further, FIG. 32A shows a state in this case.

Still further, in the set-top box 200-1, in a case where the display200-2A does not have the function of the unblending process (reverseblending process), the post-processor 205 therein executes theunblending process (reverse blending process) on the image data itemsP′(N) and P(N+1). With this, the unblended moving-image data item R atthe high frame rate is obtained. Then, from the set-top box 200-1, thismoving-image data item R is transmitted to the display 200-2A via theHDMI transmission path. FIG. 32B shows a state in this case.

Further, in the set-top box 200-1, in a case where a display 200-2Bcompatible with the moving-image data item at the normal frame rate (60Hz) is connected thereto, the decoder 204 executes the decoding processon the stream STb. With this, the image data item P′(N) of each of theframes corresponding to the normal frame rate is obtained. In addition,from the set-top box 200-1, these image data items P′(N) are transmittedto the display 200-2B via the HDMI transmission path.

As described above, in the set-top box 200-1, the image data items P′(N)and P(N+1) that have been subjected to the blending process, theinformation items of the blending rates (coefficient sets) of theframes, and the range information item of each of the image data itemsare transmitted, via the HDMI transmission path, to the display 200-2Acompatible with the moving-image data item at the high frame rate (120Hz) and has the function of the unblending process (reverse blendingprocess).

In this case, the information items of the blending rates (coefficientsets) and the range information item are transmitted, for example, undera state of being inserted into blanking periods of the image data itemsP′(N) and P(N+1). In this case, newly defined “HFR Blending InfoFrames”are used.

FIG. 33 shows a structural example (syntax) of the “HFR BlendingInfoFrame,” and FIG. 34 shows contents of main information items(semantics) in this structural example. First three bytes of thisInfoframe correspond to a header part in which information items of“InfoFrame Type,” “Version Number,” and byte lengths of “Data Bytes” arearranged.

A three-hit information item of “Frame Rate” is arranged from a seventhhit to a fifth bit of “Data Byte 1.” This three-bit information itemindicates a frame rate. For example, “3” indicates 120 Hz. Further, aone-bit information item of “Blending_flag” is arranged in a fourth bitof “Data Byte 1.” This information item indicates whether or not theprocess of blending with an peripheral image data item is applied. Forexample, “0” indicates “Not Applied,” and “1” indicates “Applied.”

Still further, a one-bit information item of “Synchronized Frame (SF)”is arranged in a zeroth bit of “Data Byte 1.” This information itemindicates whether nor not a process of synchronization with a next videoframe is essential. For example, “0” indicates that the process ofsynchronization with the next video frame is inessential, and “1”indicates that the process of synchronization with the next video frameis essential.

Yet further, an eight-bit information item of “bit_depth_information” isarranged in Data Byte 2. This information item indicates a bit width ofan encoded pixel. For example, “0” indicates 8 bits, “1” indicates 10bits, “2” indicates 12 bits, and “3” indicates 16 bits.

Yet further, the sixteen-bit information items “Range_limit_high_value”are arranged in “Data Byte 3” and “Data Byte 4.” These information itemsindicate the level value of the upper limit of thenormal-blending-process target range. Yet Further, the sixteen-bitinformation items “Range_limit_low_value” are arranged in “Data Byte 5”and “Data Byte 6.” These information items indicate the level value ofthe lower limit of the normal-blending-process target range.

Yet further, the eight-bit information item of “Blending_mode” isarranged in “Data Byte 7.” This information item indicates the modes ofthe blending processes. For example, “0x0” indicates the mode0, that is,the mode of executing only the normal blending process in related art.Further, for example, “0x1” indicates the mode1, that is, the mode ofexecuting the blending processes including the special blending processbased on the determination of the pixels in the picture “N.” Stillfurther, for example, “0x2” indicates the mode2, that is, the mode ofexecuting the blending processes including the special blending processbased on the determination of the pixels in each of the pictures “N” and“N+1.”

Yet further, the eight-bit information item of“Type0_blending_coefficient_a” is arranged in “Data Byte 8.” Thisinformation item indicates the coefficient “a” (coefficient for thebase-layer pixels) in the “Type1 blending process” being the normalblending process. Yet further, the eight-bit information item of“Type0_blending_coefficient_b” is arranged in “Data Byte 9.” Thisinformation item indicates the coefficient “b” (coefficient for theenhanced pixels) in the “Type0 blending process” being the normalblending process.

Yet further, the eight-hit information item of“Type1_blending_coefficient_c” is arranged in “Data Byte 10.” Thisinformation item indicates the coefficient “c” (coefficient for thebase-layer pixels) in the “Type1 blending process” being the specialblending process. Yet further, the eight-bit information item of“Type1_blending_coefficient_d” is arranged in “Data Byte 11.” Thisinformation item indicates the coefficient “d” (coefficient for theenhanced pixels) in the “Type1 blending process” being the specialblending process.

Yet further, the eight-bit information item of“Type2_blending_coefficient_e” is arranged in “Data Byte 12.” Thisinformation item indicates the coefficient “e” (coefficient for thebase-layer pixels) in the “Type2 blending process” being the specialblending process. Yet further, the eight-bit information item of“Type2_blending_coefficient_f” is arranged in “Data Byte 13.” Thisinformation item indicates the coefficient “f” (coefficient for theenhanced pixels) in the “Type2 blending process” being the specialblending process.

FIG. 35 shows a configuration example of the set-top box 200-1. In FIG.35 , the parts corresponding to those in FIG. 23 are denoted by the samereference symbols. This set-top box 200-1 includes a control unit 201-1,the receiving unit 202, the demultiplexer 203, the decoder 204, thepost-processor 205, and an HDMI transmitting unit 208.

The control unit 201-1 controls operations of the units in the set-topbox 200-1. The receiving unit 202 acquires the transport stream TS bydemodulating the RF-modulated signal received via the receiving antenna,and transmits the transport stream TS to the demultiplexer 203.

Depending on whether or not the display 200-2 is compatible with themoving-image data item at the high frame rate (120 Hz), thedemultiplexer 203 extracts, by filtering the PIDs, both the base streamSTb and the enhanced stream STe, or only the base stream STb.

When the base stream STb and the enhanced stream STe are extracted inthe demultiplexer 203, the decoder 204 executes the decoding process onthe base stream STb so as to generate the image data item P′(N) of eachof the frames corresponding to the normal frame rate, and executes thedecoding process on the enhanced stream STe so as to generate the imagedata item P(N+1) of each of the enhanced frames corresponding to thehigh frame rate.

Further, at this time, the decoder 204 extracts the parameter set andthe SEI that are inserted into each of the access units of each of thebase stream STb and the enhanced stream STe, and transmits theseinformation items to the control unit 201-1. In this case, the“Blend_and_range_information SEI messages” (refer to FIG. 19 ) eachcontaining the information item of the blending rate (coefficient set)of each of the frames, and the range information item of the image dataitem are also extracted.

With this, the control unit 201-1 is allowed to recognize theinformation items of the blending rates (coefficient sets) of each ofthe frames and the range information item of each of the image dataitems, and hence is allowed to appropriately determine which type of theunblending processes (reverse blending processes) to apply on thepixel-by-pixel basis when the unblending process (reverse blendingprocess) is executed in the post-processor 205. In addition, the controlunit 201-1 is allowed to appropriately set the filtering coefficients inaccordance with the types the unblending processes. Further, the controlunit 201-1 is allowed to obtain, from the “Blend_and_range_informationSEI message,” various information items that are arranged in the “HFRBlending InfoFrame” at the time of transmitting the “HFR BlendingInfoFrame” to the display 200-2.

Further, in the case where only the base stream STb is extracted in thedemultiplexer 203, the decoder 204 executes the decoding process on thisbase stream STb. With this, the image data item P′(N) of each of theframes corresponding to the normal frame rate is obtained.

In the case where the display 200-2 is compatible with the moving-imagedata item at the high frame rate, and does not have the function of theunblending process, the post-processor 205 executes the unblendingprocesses on the image data items P′(N) and P(N+1) obtained in thedecoder 204. With this, the unblended moving-image data item R at thehigh frame rate is obtained.

The HDMI transmitting unit 208 transmits, by communication using HDMI,an uncompressed moving-image data item to the display 200-2 via the HDMItransmission path. Note that, in the case where the display 200-2 iscompatible with the moving-image data item at the high frame rate, anddoes not have the function of the un-blending process, the moving-imagedata item R unblended in the post-processor 205 is transmitted to thedisplay 200-2 via the HDMI transmission path.

Further, in the case where the display 200-2 is compatible with themoving-image data item at the high frame rate, and has the function ofthe unblending process, the image data items P′(N) and P(N+1) obtainedin the decoder 204 are transmitted to the display 200-2 via the HDMItransmission path. In this case, the unblending process is executed onthe display 200-2 side, and hence the “HFR Blending InfoFrames”containing the information items of the blending rates (refer to FIG. 33) are transmitted under the state of being inserted into the blankingperiods of the image data items of the frames of the image data itemsP′(N) and P(N+1).

Still further, in the case where the display 200-2 is compatible withthe moving-image data item at the normal frame rate, the image data itemP′(N) of each of the frames corresponding to the normal frame rate,which is obtained in the decoder 204, is transmitted to the display200-2 via the HDMI transmission path.

FIG. 36 shows a configuration example of the display 200-2A compatiblewith the moving-image data item at the high frame rate. In FIG. 36 , theparts corresponding to those in FIG. 23 are denoted by the samereference symbols. This display 200-2A includes a control unit 201-2A,an HDMI receiving unit 219, the post-processor 205, the MCFI unit 206,and the panel display unit 207. Note that the post-processor 205 may beomitted.

The control unit 201-2A controls operations of the units in the display200-2A. The HDMI receiving unit 219 receives, by the communication usingHDMI, the uncompressed moving-image data item at the high frame ratefrom the set-top box 200-1 via the HDMI transmission path. Note that, inthe case where the post-processor 205 is omitted, the unblendedmoving-image data item R is received.

Meanwhile, in the case where the post-processor 205 is provided, theimage data items P′(N) and P(N+1) that have been subjected to theblending process are received. In this case, the “HFR BlendingInfoFrame” inserted into the blanking period of each of the image dataitems P′(N) and P(N+1) (refer to FIG. 33 ) is extracted and transmittedto the control unit 201-2A. With this, the control unit 201-1A isallowed to recognize the information items of the blending rates(coefficient sets) of each of the frames and the range information itemof each of the image data items, and hence is allowed to appropriatelydetermine which type of the unblending processes (reverse blendingprocesses) to apply on the pixel-by-pixel basis when the unblendingprocess (reverse blending process) is executed in the post-processor205. In addition, the control unit 201-1A is allowed to appropriatelyset the filtering coefficients in accordance with the types theunblending processes.

Under control by the control unit 201-2A, the post-processor 205executes the un-blending processes (reverse blending processes) on theimage data items P′(N) and P(N+1) received by the HDMI receiving unit219. With this, the unblended moving-image data item R at the high framerate is obtained.

The MCFI unit 206 executes the motion-compensated frame interpolationprocess on the moving-image data item R at the high frame rate, which isreceived by the HDMI receiving unit 219 or obtained in thepost-processor 205. With this, a moving-image data item at a much higherrate is obtained. Note that this MCFI unit 206 may be omitted. The paneldisplay unit 207 displays images of the moving-image data item R at thehigh frame rate, which is received by the HDMI receiving unit 219 orobtained in the post-processor 205, or images of the moving-image dataitem increased in frame rate in the MCFI unit 206.

FIG. 37 shows a configuration example of the display 200-2B compatiblewith the moving-image data item at the normal frame rate. In FIG. 37 ,the parts corresponding to those in FIG. 23 are denoted by the samereference symbols. This display 200-2B includes a control unit 201-2B,an HDMI receiving unit 219B, the MCFI unit 206B, and the panel displayunit 207B.

The control unit 201-2B controls operations of the units in the display200-2B. The HDMI receiving unit 219B receives, by the communicationusing HDMI, an uncompressed moving-image data item P′(N) at the normalframe rate from the set-top box 200-1 via the HDMI transmission path.

The MCFI unit 206B executes the motion-compensated frame interpolationprocess on the moving-image data item P′(N) at the normal frame rate,which are received by the HDMI receiving unit 219B. With this, amoving-image data item at a higher rate is obtained. Note that this MCFIunit 206B may be omitted. The panel display unit 207B displays images ofthe moving-image data item at the normal frame rate, which is receivedby the HDMI receiving unit 219B, or images of the moving-image data itemincreased in frame rate in the MCFI unit 206B.

As described hereinabove, in the transmitting-and-receiving system 10Ashown in FIG. 29 , in the case where the image data items P′(N) andP(N+1) that have been subjected to the blending process are transmittedto the display 200-2, the “HFR Blending InfoFrames” containing theinformation items of the blending rates of the frames (refer to FIG. 33) are transmitted together. Thus, in the display 200-2, the un blendingprocess is executed on the image data items P′(N) and P(N+1) on thebasis of the information items of the blending rates of the frames andthe range information item of each of the image data items. With this,the unblended moving-image data item can be easily obtained, and thismoving image can be satisfactorily displayed.

3. Third Embodiment Transmitting-and-Receiving System

In the embodiments described hereinabove, in the transmitting apparatus100, the blending process is executed on the moving-image data item P atthe high frame rate, and the image data items P′(N) and P(N+1) after theblending process are transmitted therefrom. However, in the case wherethe receiving side is compatible with the moving-image data item at thenormal frame rate, it is also conceivable to transmit the moving-imagedata item P at the high frame rate as it is from the transmitting side,and to convert the frame rate by executing the blending process on thereceiving side.

FIG. 38 shows a configuration example of a transmitting-and-receivingsystem 10′. This transmitting-and-receiving system 10′ includes atransmitting apparatus 100′, a set-top box (STB) 200′-1, and a display200′-2. The set-top box (STB) 200′-1 and the display 200′-2 areconnected to each other via the HDMI.

The transmitting apparatus 100′ transmits the transport stream TS as acontainer via a broadcast wave. This transport stream TS contains avideo stream that is obtained by encoding a moving-image data item atthe high frame rate of, for example, 120 Hz or 240 Hz, morespecifically, at 120 Hz in this embodiment.

The set-top box 200′-1 receives the above-mentioned transport stream TSthat is transmitted via the broadcast wave from the transmittingapparatus 100′. In a case where the display 200′-2 is compatible withthe moving-image data item at the high frame rate (120 Hz), the set-topbox 200′-1 decodes the video stream contained in the transport streamTS. With this, the moving-image data item at the high frame rate isobtained and transmitted to the display 200′-2.

Meanwhile, in a case where the display 200′-2 is compatible with themoving-image data item at the normal frame rate (60 Hz), the set-top box200′-1 decodes the video stream contained in the transport stream TS.The moving-image data item at the high frame rate (120 Hz) obtained inthis way is subjected to rate conversion. With this, the moving-imagedata item at the normal frame rate is obtained and transmitted to thedisplay 200′-2. In this case, at the time of acquiring the image dataitems corresponding to the normal frame rate, the process of blendingthe image data items of peripheral frames is executed such that astroboscopic effect and the like are restrained.

FIG. 39 schematically shows processes by the transmitting apparatus100′, the set-top box 200′-1, and the displays 200′-2 (200′-2A and200′-2B). The moving-image data item Va at a higher frame rate, which isoutput from the camera (imaging apparatus) 81, is transmitted to the HFRprocessor 82. With this, the moving-image data item Vb at the high framerate (120 Hz) is obtained. This moving-image data item Vb is input asthe moving-image data item P to the transmitting apparatus 100′. In thetransmitting apparatus 100′, the encoder 103 executes the encodingprocess on the moving-image data item P. With this, a video stream ST isobtained. This video stream ST is transmitted from the transmittingapparatus 100′ to the set-top box 200′-1.

In the set-top box 200′-1, the decoder 204 executes the decoding processon the video stream ST. With this, the moving-image data item P at thehigh frame rate is obtained. In the set-top box 200′-1, thismoving-image data item P is transmitted as it is, via the HDMItransmission path, to the display 200′-2A compatible with themoving-image data item at the high frame rate (120 Hz).

Further, in the set-top box 200′-1, a post-processor 209B executes theblending process on, among the image data items of the frames of themoving-image data item P, the image data item of each of the framescorresponding to the normal frame rate. With this, the image data itemP′(N) of each of the frames corresponding to the normal frame rate isobtained. In the set-top box 200′-1, these image data items P′(N) aretransmitted, via the HDMI transmission path, the display 200′-2Bcompatible with the moving-image data item at the normal frame rate (60Hz).

In the display 200′-2A, the moving-image data item P is used as it is asa displaying moving-image data item, or converted to the same by beingincreased in frame rate through the frame interpolation in themotion-compensated frame interpolation (MCFI) unit 206. Further, in thedisplay 200′-2B, the moving-image data item including the image dataitems P′(N) is used as it is as a displaying moving-image data item, orconverted to the same by being increased in frame rate through the frameinterpolation in the motion-compensated frame interpolation (MCFI) unit206B.

FIG. 40 shows a configuration example of the transmitting apparatus100′. In FIG. 40 , the parts corresponding to those in FIG. 11 aredenoted by the same reference symbols. This transmitting apparatus 100′includes the control unit 101, the encoder 103, the multiplexer 104, andthe transmitting unit 105. The control unit 101 controls operations ofthe units in the transmitting apparatus 100′.

In the encoder 103, the encoding process is executed on the moving-imagedata item P at the high frame rate. With this, the video stream ST isgenerated. In this case, the predictive encoding processes such asH.264/AVC and H.265/HEVC are executed on the moving-image data item P.

In the multiplexer 104, the video stream ST generated in the encoder 103is packetized into the packetized elementary stream (PES) packets, andfurther into the transport packets to be multiplexed. In this way, thevideo stream ST as a multiplexed stream is obtained. In the transmittingunit 105, this transport stream TS is modulated, for example, inaccordance with the modulation scheme suited to the broadcasting, suchas QPSK and OFDM, and the RF signal is transmitted via the transmittingantenna.

FIG. 41 shows a configuration example of the set-top box 200′-1. In FIG.41 , the parts corresponding to those in FIG. 23 and FIG. 35 are denotedby the same reference symbols. This set-top box 200′-1 includes acontrol unit 201′-1, the receiving unit 202, the demultiplexer 203, thedecoder 204, the post-processor 209B, and the HDMI transmitting unit208.

The control unit 201′-1 controls operations of the units in the set-topbox 200′-1. In the receiving unit 202, the transport stream TS isacquired by demodulating the RF-modulated signal received via thereceiving antenna. In the demultiplexer 203, the video stream ST isextracted from the transport stream TS by filtering the PIDs, and thensupplied to the decoder 204.

In the decoder 204, the decoding process is executed on the video streamST. With this, the moving-image data item P at the high frame rate isobtained. Further, in the post-processor 209B, among the image dataitems of the frames of the moving-image data item P, the image data itemof each of the frames corresponding to the normal frame rate issubjected to the blending process. With this, the image data item P′(N)of each of the frames corresponding to the normal frame rate isobtained.

In the HDMI transmitting unit 208, by the communication using HDMI, theuncompressed moving-image data items are transmitted to the displays200′-2A and 200′-2B via the HDMI transmission path. In this case, themoving-image data item P obtained in the decoder 204 is transmitted, viathe HDMI transmission path, to the display 200′-2A compatible with themoving-image data item at the high frame rate (120 Hz). Meanwhile, theimage data items P′(N) obtained in the post-processor 209B aretransmitted, via the HDMI transmission path, to the display 200′-2Bcompatible with the moving-image data item at the normal frame rate (60Hz).

FIG. 42 shows a configuration example of the post-processor 209B.Although not described in detail, this post-processor 209B has the sameconfiguration and performs the same operations as those of thepre-processor 102 shown in FIG. 12 except that the output system for theimage data items P(N+1) is omitted. With this, the image data itemsP′(N) are obtained. Note that this post-processor 209B is different fromthe pre-processor 102 of FIG. 12 also in that the pixel processing unit120 may be bypassed in a case where it is unnecessary to take intoconsideration the determination based on the image data items P′(N) asto whether or not the pixel values are within one of thespecial-blending-process target ranges.

Further, FIG. 43 shows another configuration example of thepost-processor 209B. Although not described in detail, thispost-processor 209B has the same configuration and performs the sameoperations as those of the pre-processor 102 shown in FIG. 16 exceptthat the output system for the image data items P(N+1) is omitted. Withthis, the image data items P′(N) are obtained. Note that thispost-processor 209B is different from the pre-processor 106 of FIG. 12also in that the pixel processing unit 120 may be bypassed in the casewhere it is unnecessary to take into consideration the determinationbased on the image data items P′(N) as to whether or not the pixelvalues are within one of the special-blending-process target ranges.Note that the information item of “Blend_and_range_information SEImessage” need not be added to the output of the post-processor 209B.

FIG. 44 shows a configuration example of the display 200′-2A compatiblewith the moving-image data item at the high frame rate (120 Hz). Thisdisplay 200′-2A includes a control unit 201′-2A, the HDMI receiving unit219, the MCFI unit 206, and the panel display unit 207.

The control unit 201′-2A controls operations of the units in the display200′-2A. The HDMI receiving unit 219 receives, by the communicationusing HDMI, an uncompressed moving-image data item P at the high framerate from the set-top box 200′-1 via the HDMI transmission path.

In the MCFI unit 206, the motion-compensated frame interpolation processis executed on the moving-image data item P at the high frame rate,which is received by the HDMI receiving unit 219. With this, amoving-image data item at a higher rate is obtained. Note that this MCFIunit 206 may be omitted. In the panel display unit 207, images of themoving-image data item P at the high frame rate, which is received bythe HDMI receiving unit 219, or images of the moving-image data itemincreased in frame rate in the MCFI unit 206 are displayed.

FIG. 45 shows a configuration example of the display 200′-2B compatiblewith the moving-image data item at the normal frame rate (60 Hz). Thisdisplay 200′-2B includes a control unit 201′-2B, the HDMI receiving unit219B, the MCFI unit 206B, and the panel display unit 207B. This display200′-2B has the same configuration as that of the display 200-2Bdescribed above with reference to FIG. 37 , and hence descriptionthereof is omitted.

As described hereinabove, in the transmitting-and-receiving system 10′shown in FIG. 38 , among the image data items of the frames of themoving-image data item P at the high frame rate (120 Hz), the image dataitem of each of the frames corresponding to the normal frame rate issubjected to the blending process by the post-processor 209B. With this,the image data item P′(N) of each of the frames corresponding to thenormal frame rate is obtained. With this, the display 200′-2B, which iscompatible with the moving-image data item at the normal frame rate (60Hz) and to which these image data items P′(N) are supplied, is allowedto smoothly display the images of the moving image. Further, theimage-quality problem as a result of the frame interpolation processincluding the low-load calculation in the display process can beavoided.

Further, by blending the image data item P(N) of each of the framescorresponding to the normal frame rate with the image data item P(N+1)at the blending rate in accordance with the data level, the image dataitem P′(N) of each of the frames corresponding to the normal frame rateis obtained. Thus, the original texture of the images, such as the highdynamic range (HDR) effect, can be prevented from being impaired by theblending processes.

4. Modification

Note that, as a general pattern of the blending process, the blendingprocess need not necessarily be executed only on the picture P(N) of thepictures P(N) and P(N+1), and may be executed on the picture P(N+1).

In that case, the blending rates in the blending of the image data itemP(N) and blending rates in the blending of the image data item P(N+1)may be set independently of each other. The image data items P′(N) andP′(N+1) are expressed as follows.P′(N)=A*P(N)+B*P(N+1)P′(N+1)=a*P(N)+b*P(N+1)

where “A” is “blending_coef_a” for the image data item P(N) in the casewhere the image data item P(N) is a blending target, “B” is“blending_coef_b” for the image data item P(N+1) in the same case, “a”is “blending_coef_a” for the image data item P(N) in the case where theimage data item P(N+1) is a blending target, and “b” is“blending_coef_b” for the image data item P(N+1) in the same case.

In the pre-processor (Preproc), these relationships are represented alsoby the following general matrix.

$\begin{matrix}{\begin{bmatrix}{P^{\prime}(N)} \\{P^{\prime}\left( {N + 1} \right)}\end{bmatrix} = {\begin{bmatrix}A & B \\a & b\end{bmatrix}*\begin{bmatrix}{P(N)} \\{P\left( {N + 1} \right)}\end{bmatrix}}} & \left\lbrack {{Math}.1} \right\rbrack\end{matrix}$

Further, in the post-processor (Postproc), the relationships arerepresented as follows.

$\begin{matrix}{\left. \lbrack\begin{matrix}{P^{''}(N)} \\{P^{''}\left( {N + 1} \right)}\end{matrix} \right\rbrack = {\begin{bmatrix}A & B \\a & b\end{bmatrix}^{- 1}*\begin{bmatrix}{P^{\prime}(N)} \\{P^{\prime}\left( {N + 1} \right)}\end{bmatrix}}} & \left\lbrack {{Math}.2} \right\rbrack\end{matrix}$

“Blending_coef_c,” “Blending_coef_d,” “Blending_coef_e,” and“Blending_coef_f” also can be expressed in the same way. Note thatflowcharts are the same as those in the cases where P(N) is a blendingtarget except that P(N) and P(N+1) are replaced with each other.

FIG. 46 schematically illustrates an example of the blending on thetransmitting side and the unblending on the receiving side in thegeneral pattern of the blending process. This example corresponds to theexample in (b) of FIG. 2 , specifically, the picture “N” and the picture“N+1” form a frame pair, and the picture “N+2” and the picture “N+3”form another frame pair. Note that, in the illustrated example, theobjects Oa and Ob are static objects, and the object Oc is a movingobject.

In each of the frame pairs, by the blending process on the transmittingside, an image data item of a first frame, specifically, an image dataitem of a frame of the base stream, is blended at a predeterminedblending rate with an image data item of an enhanced frame (blendedstate). Similarly, an image data item of a frame subsequent thereto,specifically, the image data item of the frame subsequent thereto of theenhanced stream is blended at a predetermined blending rate with theimage data item of the base frame (blended state).

FIG. 47A and FIG. 47B each show a method of arranging SEIs in thegeneral pattern of the blending process. FIG. 47A shows a first method.This first method is a method of arranging an SEI of a group (basesublayer) in which P(N) is a blending target, and an SEI of a group(enhanced sublayer) in which P(N+1) is a blending target respectivelyinto these sublayers.

FIG. 47B shows a second method. This second method is a method ofarranging the SEI of the group (base sublayer) in which P(N) is ablending target, and the SEI of the group (enhanced sublayer) in whichP(N+1) is a blending target into one of these sublayers. In order thatthe blending target can be identified in either one of the methods,elements “blending target” are defined in a syntax of each of the SEIs.

FIG. 48 shows a structural example of “Blend_and_range_information SEImessage” in the general pattern of the blending process. This structuralexample is the same as the structural example shown in FIG. 19 exceptthat the element “blending_target” is newly defined. For example, “0”indicates the base sublayer, and “1” indicates a first enhancedsublayer. FIG. 49 shows a structural example of “HFR Blending InfoFrame”in the general pattern of the blending process. This structural exampleis the same as the structural example shown in FIG. 33 except that theelement “blending_target” is newly defined.

Note that the combinations of the frame rates are not limited to thoseof the examples in the embodiments described hereinabove, that is, notlimited to the high frame rate of 120 Hz or 240 Hz and the normal framerate of 60 Hz. For example, there may be employed combinations of 100 Hzor 200 Hz and 50 Hz. Further, description of the embodiments, which ismade hereinabove with a focus on the luminance, may be made with a focuson colors. In that case, the similar processes are executed in an RGBdomain.

Further, the configuration of the transmitting-and-receiving system towhich the present technology is applicable is not limited to those ofthe systems in the embodiments described hereinabove, that is, notlimited to the transmitting-and-receiving system 10 including thetransmitting apparatus 100 and the television receiver 200, or to thetransmitting-and-receiving system 10A including the transmittingapparatus 100, the set-top box 200-1, and the display 200-2.

Still further, in the examples in the embodiments described hereinabove,the container is the transport stream (MPEG-2 TS). However, the presenttechnology is applicable also to systems that perform distribution toreceiving terminals by utilizing networks such as the Internet.Containers in other formats such as MP4 are frequently used indistribution via the Internet. More specifically, as examples of thecontainers, there may be mentioned containers in various formats such asthe transport stream (MPEG-2 TS) and MPEG Media Transport (MMT) that areemployed as digital broadcasting standards, and ISOBMFF (MP4) used inthe distribution via the Internet.

The present technology may also provide the following configurations.

(1) A transmission apparatus comprising:

-   -   circuitry configured to    -   perform processing of mixing, at a mixing rate, pixels of each        frame of first video data with pixels of one or more peripheral        frames of the first video data and obtain second video data at a        first frame rate, wherein the mixing rate for each pixel of the        respective frame of the first video data is based on a luminance        value of the respective pixel,    -   the second video data including frames corresponding to a second        frame rate that is lower than the first frame rate, the frames        corresponding to the second frame rate being mixed with the        peripheral frames, and    -   the circuitry is further configured to    -   encode the frames corresponding to the second frame rate to        obtain a basic stream and encode remaining frames of the second        video data to obtain an extended stream,    -   insert information about the mixing rate for each pixel of the        respective frame of the first video data into the basic stream        and the extended stream in association with the respective        frame, and    -   transmit the basic stream and the extended stream into which the        information about the mixing rate has been inserted.

(2) The transmission apparatus according to Item (1), wherein theinformation about the mixing rate for each pixel of the respective frameof the first video data includes plural mixing rates and a correspondingluminance range for at least one of the mixing rates.

(3) The transmission apparatus according to Item (1), wherein the basicstream and the extended stream have a Network Abstraction Layer (NAL)unit structure, and

-   -   the circuitry is configured to insert a Supplemental Enhancement        Information (SEI) NAL unit with the information about the mixing        rate into the basic stream and the extended stream.

(4) The transmission apparatus according to Item (1), wherein thecircuitry is configured to determine, when performing the processing ofmixing the pixels of each frame of the first video data with the pixelsof the one or more peripheral frames of the first video data, the mixingrate for each pixel of the respective frame of the first video databased on a luminance value of the respective pixel.

(5) The transmission apparatus according to Item (1), wherein thecircuitry is configured to determine, when performing the processing ofmixing the pixels of each frame of the first video data with the pixelsof the one or more peripheral frames of the first video data, the mixingrate for each pixel of the respective frame of the first video databased on a luminance value of the respective pixel, and based on theluminance values of the pixels of the one or more peripheral frames.

(6) The transmission apparatus according to Item (2), wherein theinformation about the mixing rate for each pixel of the respective frameof the first video data includes a first luminance threshold and asecond luminance threshold, the first luminance threshold and the secondluminance threshold defining the corresponding luminance range for atleast one of the mixing rates.

(7) The transmission apparatus according to Item (1), wherein the firstframe rate is 120 Hz or 240 Hz, and the second frame rate is 60 Hz.

(8) A transmission method comprising:

-   -   performing, by circuitry, processing of mixing, at a mixing        rate, pixels of each frame of first video data with pixels of        one or more peripheral frames of the first video data and        obtaining second video data at a first frame rate, wherein the        mixing rate for each pixel of the respective frame of the first        video data is based on a luminance value of the respective        pixel,    -   the second video data including frames corresponding to a second        frame rate that is lower than the first frame rate, the frames        corresponding to the second frame rate being mixed with the        peripheral frames, and    -   the transmission method further includes    -   encoding, by the circuitry, the frames corresponding to the        second frame rate to obtain a basic stream and encoding        remaining frames of the second video data to obtain an extended        stream,    -   inserting, by the circuitry, information about the mixing rate        for each pixel of the respective frame of the first video data        into the basic stream and the extended stream in association        with the respective frame, and    -   transmitting, by the circuitry, the basic stream and the        extended stream into which the information about the mixing rate        has been inserted.

(9) A reception apparatus comprising:

-   -   circuitry configured to receive a basic stream and an extended        stream, which are obtained by    -   performing processing of mixing, at a mixing rate, pixels of        each frame of first video data with pixels of one or more        peripheral frames of the first video data and obtaining second        video data at a first frame rate, wherein    -   the mixing rate for each pixel of the respective frame of the        first video data is based on a luminance value of the respective        pixel, and    -   the second video data including frames corresponding to a second        frame rate that is lower than the first frame rate, the frames        corresponding to the second frame rate are mixed with the        peripheral frames,    -   encoding the frames corresponding to the second frame rate to        obtain the basic stream, and    -   encoding remaining frames of the second video data to obtain the        extended stream, information about the mixing rate for each        pixel of the respective frame of the first video data is        included in the basic stream and the extended stream in        association with the respective frame, and    -   the reception apparatus further includes circuitry configured        to, based on a frame rate capability of a display connected to        the reception apparatus,    -   decode the basic stream to obtain frames at the second frame        rate or    -   decode the basic stream and the extended stream to obtain the        second video data, and obtain mixing-released video data at the        first frame rate by performing back mixing processing on the        second video data on a basis of the information about the mixing        rate.

(10) The reception apparatus according to Item (9), wherein theinformation about the mixing rate for each pixel of the respective frameof the first video data includes plural mixing rates and a correspondingluminance range for at least one of the mixing rates, and

-   -   the circuitry is configured to perform back mixing processing        based on the plural mixing rates and the corresponding luminance        range for at least one of the mixing rates.

(11) A reception method comprising:

-   -   receiving, by circuitry, a basic stream and an extended stream,        which are obtained by performing processing of mixing, at a        mixing rate, pixels of each frame of first video data with        pixels of one or more peripheral frames of the first video data        and obtaining second video data at a first frame rate, wherein        the mixing rate for each pixel of the respective frame of the        first video data is based on a luminance value of the respective        pixel,    -   the second video data including frames corresponding to a second        frame rate that is lower than the first frame rate, the frames        corresponding to the second frame rate are mixed with the        peripheral frames,    -   encoding the frames corresponding to the second frame rate to        obtain the basic stream, and    -   encoding remaining frames of the second video data to obtain the        extended stream, information about the mixing rate for each        pixel of the respective frame of the first video data is        included in the basic stream and the extended stream in        association with the respective frame, and    -   the reception method further includes, based on a frame rate        capability of a display connected to the reception apparatus,    -   decoding, by the circuitry, the basic stream to obtain frames at        the second frame rate, or    -   decoding the basic stream and the extended stream to obtain the        second video data, and obtaining mixing-released video data at        the first frame rate by performing back mixing processing on the        second video data on a basis of the information about the mixing        rate.

(12) A reception apparatus comprising:

-   -   circuitry configured to    -   acquire second video data obtained by performing processing of        mixing, at a mixing rate, pixels of each frame of first video        data with pixels of one or more peripheral frames of the first        video data, wherein the mixing rate for each pixel of the        respective frame of the first video data is based on a luminance        value of the respective pixel; and transmit the second video        data and information about the mixing rate for each pixel of the        respective frame of the first video data to an external device        via a transfer path, the information about the mixing rate for        each pixel of the respective frame of the first video data        includes plural mixing rates and a corresponding luminance range        for at least one of the mixing rates.

(13) The reception apparatus according to Item (12),

-   -   wherein the circuitry is configured to respectively insert the        information about the mixing rate for each pixel of the        respective frame into a blanking period of the respective frame        of the second video data and transmit the second video data.

(14) The reception apparatus according to Item (12), wherein thecircuitry is further configured to perform back mixing processing oneach frame of the second video data on a basis of the information aboutthe mixing rate to obtain third video data, wherein the circuitry isconfigured to transmit the third video data instead of the second videodata when the external device does not have a function of the backmixing processing.

(15) The reception apparatus according to Item (12),

-   -   wherein the second video data has a first frame rate,    -   the second video data including frames corresponding to a second        frame rate that is lower than the first frame rate, the frames        corresponding to the second frame rate are mixed with the        peripheral frames, and    -   the circuitry is further configured to transmit fourth video        data that includes the frames corresponding to the second frame        rate instead of the second video data when a frame rate at which        display is able to be performed by the external device is the        second frame rate.

(16) A reception method comprising:

-   -   acquiring, by circuitry, second video data obtained by        performing processing of mixing, at a mixing rate, pixels of        each frame of first video data with pixels of one or more        peripheral frames of the first video data, wherein the mixing        rate for each pixel of the respective frame of the first video        data is based on a luminance value of the respective pixel; and    -   transmitting, by the circuitry, the second video data and        information about the mixing rate for each pixel of the        respective frame of the first video data to an external device        via a transfer path, the information about the mixing rate for        each pixel of the respective frame of the first video data        includes plural mixing rates and a corresponding luminance range        for at least one of the mixing rates.

(17) A reception apparatus comprising:

-   -   circuitry configured to    -   receive second video data obtained by performing processing of        mixing, at a mixing rate, pixels of each frame of first video        data with pixels of one or more peripheral frames of the first        video data, and information about a mixing rate for each pixel        of the respective frame of the first video data from an external        device via a transfer path, wherein the mixing rate for each        pixel of the respective frame of the first video data is based        on a luminance value of the respective pixel; and    -   obtain mixing-released video data by performing back mixing        processing on each frame of the second video data on a basis of        the information about the mixing rate, the information about the        mixing rate for each pixel of the respective frame of the first        video data includes plural mixing rates and a corresponding        luminance range for at least one of the mixing rates.

(18) A reception method comprising:

-   -   receiving, by circuitry, second video data obtained by        performing processing of mixing, at a mixing rate, pixels of        each frame of first video data with pixels of one or more        peripheral frames of the first video data, and information about        a mixing rate for each pixel of the respective frame of the        first video data from an external device via a transfer path,        wherein the mixing rate for each pixel of the respective frame        of the first video data is based on a luminance value of the        respective pixel; and    -   obtaining, by the circuitry, mixing-released video data by        performing back mixing processing on each frame of the second        video data on a basis of the information about the mixing rate,        the information about the mixing rate for each pixel of the        respective frame of the first video data includes plural mixing        rates and a corresponding luminance range for at least one of        the mixing rates.

(19) A receiving apparatus, comprising:

-   -   a receiver configured to receive a video stream obtained by        encoding second video data at a first frame rate; and    -   circuitry configured to control    -   decoding the video stream such that the second video data at the        first frame rate is obtained, and    -   mixing, at a mixing rate, pixels of each frame of first video        data with pixels of one or more peripheral frames of the first        video data, the second video data including frames corresponding        to a second frame rate that is lower than the first frame rate,        the frames corresponding to the second frame rate are mixed with        the peripheral frames, such that a basic stream at the second        frame rate is obtained.

(20) The receiving apparatus according to claim 19, wherein the mixingrate for each pixel of the respective frame of the first video data isbased on a luminance value of the respective pixel.

A main feature of the embodiments of the present technology lies inthat, by blending, among the image data items of the frames of themoving-image data item at the high frame rate, at least the image dataitems of the frames corresponding to the normal frame rate with theimage data items of the peripheral frames, and by transmitting the imagedata items of the frames corresponding to the normal frame rate underthis state, that is, under the state of the higher shutter-opening rate,the receiver having the decoding capability to process the moving-imagedata item at the normal frame rate is allowed to smoothly display theimages of this moving image in such a manner that a stroboscopic effectis reduced (refer to FIG. 5 ). Further, another main feature lies inthat, when executing the blending processes, blending is performed atthe blending rate in accordance with the data levels and on thepixel-by-pixel basis, whereby the original texture of the images, suchas the HDR effect, can be prevented from being impaired by the blendingprocesses (refer to FIG. 12 and FIG. 16 ).

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

REFERENCE SIGNS LIST

-   -   10, 10A, 10′ Transmitting-and-receiving system    -   81 Camera    -   82 HFR processor    -   100, 100′ Transmitting apparatus    -   101 Control unit    -   102 Pre-processor    -   103 Encoder    -   104 Multiplexer    -   105 Transmitting unit    -   120 Pixel processing unit    -   121 Frame delay unit    -   122, 125, 126, 130, 131 Coefficient multiplier    -   124, 127, 132 Adding unit    -   128, 133 Switching unit    -   129 Output unit    -   200, 200A, 200B, 200′ Television receiver    -   200-1, 200′-1 Set-top box    -   200-2, 200-2A, 200-2B, 200′-2A, 200′-2B Display    -   201, 201B, 201-1, 201′-1, 201-2A, 201-2B, 201′-2A, 201′-2B        Control unit    -   202, 202B Receiving unit    -   203, 203B Demultiplexer    -   204, 204B Decoder    -   205 Post-processor    -   206, 206B MCFI unit    -   207, 207B Panel display unit    -   208 HDMI transmitting unit    -   209B Post-processor    -   219, 219B HDMI receiving unit    -   220, 221, 223, 224, 230, 231 Coefficient multiplier    -   222, 225, 232 Adding unit    -   226, 227, 233 Switching unit

The invention claimed is:
 1. A reception apparatus, comprising: adisplay; and circuitry configured to: receive a basic stream and anextended stream, the extended stream including an encoded subset offrames of first video data of a first frame rate and the basic streamincluding encoded frames of second video data of a second frame ratelower than the first frame rate, each frame of the second video datarepresenting a blending of two or more adjacent frames of the firstvideo data, obtain blending information, the blending informationproviding one of a plurality of blending rates for each pixel of arespective frame of the first video data, the blending rate being basedon a luminance value of the pixel, at least one of the plurality ofblending rates having a corresponding luminance range, decode the basicstream to obtain the second video data of the second frame rate, decodethe extended stream to obtain the subset of frames of the first videodata, obtain unblended video data at the first frame rate by performingan unblending process on the second video data on a basis of theplurality of blending rates and the at least one corresponding luminancerange, and control the display in accordance with the first video dataor the second video data.
 2. The reception apparatus according to claim1, wherein the blending information is included in at least one of thebasic stream or the extended stream.
 3. The reception apparatusaccording to claim 1, wherein the blending information includes aplurality of blending coefficients corresponding to the plurality ofblending rates.
 4. The reception apparatus according to claim 3, whereinthe blending information includes a first luminance threshold and asecond luminance threshold, the first luminance threshold and the secondluminance threshold defining the corresponding luminance range for theat least one of the plurality of blending rates.
 5. The receptionapparatus according to claim 1, wherein the circuitry is configured toperform the unblending process on the second video data using the subsetof frames of the first video data.
 6. The reception apparatus accordingto claim 1, wherein: the basic stream and the extended stream have aNetwork Abstraction Layer (NAL) unit structure, the blending informationis included in a Supplemental Enhancement Information (SEI) NAL unit,and the SEI NAL unit is included in at least one of the basic stream orthe extended stream.
 7. The reception apparatus according to claim 1,wherein the first frame rate is 120 Hz or 240 Hz, and the second framerate is 60 Hz.
 8. The reception apparatus according to claim 1, whereinat least one of the basic stream or the extended stream is received viabroadcast.
 9. A method comprising: receiving a basic stream and anextended stream, the extended stream including an encoded subset offrames of first video data of a first frame rate and the basic streamincluding encoded frames of second video data of a second frame ratelower than the first frame rate, each frame of the second video datarepresenting a blending of two or more adjacent frames of the firstvideo data, obtaining blending information, the blending informationproviding one of a plurality of blending rates for each pixel of arespective frame of the first video data, the blending rate being basedon a luminance value of the pixel, at least one of the plurality ofblending rates having a corresponding luminance range, decoding thebasic stream to obtain the second video data of the second frame rate,decoding the extended stream to obtain the subset of frames of the firstvideo data, obtaining unblended video data at the first frame rate byperforming an unblending process on the second video data on a basis ofthe plurality of blending rates and the at least one correspondingluminance range, and controlling a display in accordance with the firstvideo data or the second video data.
 10. The method according to claim9, wherein the blending information is included in at least one of thebasic stream or the extended stream.
 11. The method according to claim9, wherein the blending information includes a plurality of blendingcoefficients corresponding to the plurality of blending rates.
 12. Themethod according to claim 11, wherein the blending information includesa first luminance threshold and a second luminance threshold, the firstluminance threshold and the second luminance threshold defining thecorresponding luminance range for the at least one of the plurality ofblending rates.
 13. The method according to claim 9, wherein theunblending process is performed on the second video data using thesubset of frames of the first video data.
 14. The method according toclaim 9, wherein: the basic stream and the extended stream have aNetwork Abstraction Layer (NAL) unit structure, the blending informationis included in a Supplemental Enhancement Information (SEI) NAL unit,and the SEI NAL unit is included in at least one of the basic stream orthe extended stream.
 15. The method according to claim 9, wherein thefirst frame rate is 120 Hz or 240 Hz, and the second frame rate is 60Hz.
 16. The method according to claim 9, wherein at least one of thebasic stream or the extended stream is received via broadcast.
 17. Anon-transitory computer-readable recording medium containing computerprogram instructions, which, when executed by a processor, causes themethod according to claim 9 to be performed.